{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"../Pierian-Data-Logo.PNG\">\n",
    "<br>\n",
    "<strong><center>Copyright 2019. Created by Jose Marcial Portilla.</center></strong>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression with PyTorch\n",
    "In this section we'll use PyTorch's machine learning model to progressively develop a best-fit line for a given set of data points. Like most linear regression algorithms, we're seeking to minimize the error between our model and the actual data, using a <em>loss function</em> like mean-squared-error.\n",
    "\n",
    "<img src='../Images/linear-regression-residuals.png' width='400' style=\"display: inline-block\"><br>\n",
    "\n",
    "Image source: <a href='https://commons.wikimedia.org/wiki/File:Residuals_for_Linear_Regression_Fit.png'>https://commons.wikimedia.org/wiki/File:Residuals_for_Linear_Regression_Fit.png</a>\n",
    "\n",
    "To start, we'll develop a collection of data points that appear random, but that fit a known linear equation $y = 2x+1$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Perform standard imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn  # we'll use this a lot going forward!\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a column matrix of X values\n",
    "We can create tensors right away rather than convert from NumPy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = torch.linspace(1,50,50).reshape(-1,1)\n",
    "\n",
    "# Equivalent to\n",
    "# X = torch.unsqueeze(torch.linspace(1,50,50), dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14.,\n",
       "        15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.,\n",
       "        29., 30., 31., 32., 33., 34., 35., 36., 37., 38., 39., 40., 41., 42.,\n",
       "        43., 44., 45., 46., 47., 48., 49., 50.])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = torch.linspace(1,50,50)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.],\n",
       "        [ 2.],\n",
       "        [ 3.],\n",
       "        [ 4.],\n",
       "        [ 5.],\n",
       "        [ 6.],\n",
       "        [ 7.],\n",
       "        [ 8.],\n",
       "        [ 9.],\n",
       "        [10.],\n",
       "        [11.],\n",
       "        [12.],\n",
       "        [13.],\n",
       "        [14.],\n",
       "        [15.],\n",
       "        [16.],\n",
       "        [17.],\n",
       "        [18.],\n",
       "        [19.],\n",
       "        [20.],\n",
       "        [21.],\n",
       "        [22.],\n",
       "        [23.],\n",
       "        [24.],\n",
       "        [25.],\n",
       "        [26.],\n",
       "        [27.],\n",
       "        [28.],\n",
       "        [29.],\n",
       "        [30.],\n",
       "        [31.],\n",
       "        [32.],\n",
       "        [33.],\n",
       "        [34.],\n",
       "        [35.],\n",
       "        [36.],\n",
       "        [37.],\n",
       "        [38.],\n",
       "        [39.],\n",
       "        [40.],\n",
       "        [41.],\n",
       "        [42.],\n",
       "        [43.],\n",
       "        [44.],\n",
       "        [45.],\n",
       "        [46.],\n",
       "        [47.],\n",
       "        [48.],\n",
       "        [49.],\n",
       "        [50.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.reshape(-1,1)\n",
    "X\n",
    "# Equivalent to\n",
    "# X = torch.unsqueeze(torch.linspace(1,50,50), dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a \"random\" array of error values\n",
    "We want 50 random integer values that collectively cancel each other out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 2.],\n",
      "        [ 7.],\n",
      "        [ 2.],\n",
      "        [ 6.],\n",
      "        [ 2.],\n",
      "        [-4.],\n",
      "        [ 2.],\n",
      "        [-5.],\n",
      "        [ 4.],\n",
      "        [ 1.],\n",
      "        [ 2.],\n",
      "        [ 3.],\n",
      "        [ 1.],\n",
      "        [-8.],\n",
      "        [ 5.],\n",
      "        [ 5.],\n",
      "        [-6.],\n",
      "        [ 0.],\n",
      "        [-7.],\n",
      "        [-8.],\n",
      "        [-3.],\n",
      "        [-1.],\n",
      "        [ 2.],\n",
      "        [-6.],\n",
      "        [-3.],\n",
      "        [ 3.],\n",
      "        [ 2.],\n",
      "        [ 3.],\n",
      "        [ 4.],\n",
      "        [ 5.],\n",
      "        [ 1.],\n",
      "        [ 7.],\n",
      "        [ 6.],\n",
      "        [-1.],\n",
      "        [-6.],\n",
      "        [-5.],\n",
      "        [-3.],\n",
      "        [ 7.],\n",
      "        [ 0.],\n",
      "        [ 8.],\n",
      "        [-1.],\n",
      "        [-2.],\n",
      "        [ 2.],\n",
      "        [-8.],\n",
      "        [-1.],\n",
      "        [ 6.],\n",
      "        [-8.],\n",
      "        [-3.],\n",
      "        [-7.],\n",
      "        [-2.]])\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(71) # to obtain reproducible results\n",
    "e = torch.randint(-8,9,(50,1),dtype=torch.float)\n",
    "print(e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.)\n"
     ]
    }
   ],
   "source": [
    "print(e.sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a column matrix of y values\n",
    "Here we'll set our own parameters of $\\mathrm {weight} = 2,\\; \\mathrm {bias} = 1$, plus the error amount.<br><strong><tt>y</tt></strong> will have the same shape as <strong><tt>X</tt></strong> and <strong><tt>e</tt></strong>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([50, 1])\n"
     ]
    }
   ],
   "source": [
    "y = 2*X + 1 + e\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the results\n",
    "We have to convert tensors to NumPy arrays just for plotting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVlklEQVR4nO3dfYxcV3nH8d8PJxFLoN28bC17HWO3jRxF0GK0TdMuqogDdYCIWCiKQBRMG8l/lNJAqYmDKkWtoCyl4qVSFZQmgCtBXhSCEzVVIYqN6KvLbjbFISFKmibEixMbgQutXEjM0z/m7mSzmTu7M773nnvnfj+StTN37s6cq0z2uec85znHESEAACTpJakbAACoD4ICAKCLoAAA6CIoAAC6CAoAgK7TUjfgVJx77rmxadOm1M0AgEaZm5v7fkRM9Hqt0UFh06ZNmp2dTd0MAGgU20/mvcbwEQCgi6AAAOgiKAAAukoLCrY/Z/uo7QeXHDvb9r22H81+npUdt+2/sv2Y7W/Zfm1Z7QIA5Cuzp/AFSZctO7ZH0n0Rcb6k+7LnkvQmSedn/3ZJuqHEdgEAcpQ2+ygivmF707LDV0h6ffZ4r6SvS7o2O/630Vmd799sj9teFxFHymofABRt3/yCPvHVR/S94ye0fnxMu7dv0Y6tk436jKqnpK5d8of+aUlrs8eTkp5act7h7NiLgoLtXer0JrRx48byWgoAA9g3v6Dr7jykE8+elCQtHD+h6+48JEmF/dGu4jOSJZqzXsHA63ZHxI0RMRURUxMTPWsvAKByn/jqI90/1otOPHtSn/jqI436jKqDwjO210lS9vNodnxB0nlLztuQHQOARvje8RMDHa/rZ1QdFO6WtDN7vFPSXUuOvzubhXSxpP8mnwCgSdaPjw10vK6fUeaU1Fsk/aukLbYP275a0oykN9p+VNIbsueS9PeSHpf0mKS/kfT7ZbULAMqwe/sWjZ2+5gXHxk5fo93btzTqM8qcffSOnJcu7XFuSHpvWW0BgLItJnrLnBlUxWe4yXs0T01NBQviAcBgbM9FxFSv11jmAgDQRVAAAHQRFAAAXY3eZAcAmqyKZTEGRVAAgASqWLJiGAwfAUACVSxZMQyCAgAkUMWSFcNg+AgAElg/PqaFHgFg/fhY0lwDPQUASCBvyYpLLpjQdXce0sLxEwo9n2vYN1/NGqEEBQBIYMfWSX3sba/W5PiYLGlyfEwfe9urdeA7x5LmGhg+AoBEdmydfNGw0Adue6DnuVXlGggKADCgMsf8++UaqsDwEQAMYLG+oKwx/yqWx+6HngKARirqbn3Q9+lXX1BEb6GK5bH7ISgAaJyiqoGHeZ8q6gt65RqqwvARgMYpqhp4mPepYkvMlAgKABqnqLv1Yd4n9Zh/2Rg+AtA4Rc3QGeZ9hhnzr+NqqHkICgAaZ/f2LS/IBUjD3a0P+z6DjPnXdTXUPAwfAWicvGrgQf/IFvU+/dR1NdQ89BQANFJRM3TKnulT19VQ89BTAIASNW22EkEBAErUtNlKDB8BQIlSVygPiqAAILkmTdkcRsoK5UERFAAk1bQpm6OOoAAgqbIXmFs06r2RohAUACRVxZRNeiOrR1AAkFTRm8r06hFU1RsZBUxJBZBUkVM28zbA6RV0pPoWkKVETwFAUkVO2czrEayxdTLiRef36420NQdBUACQXFFTNvPu/E9GaOz0Nate+K7NOYgkw0e2P2D727YftH2L7Zfa3mz7oO3HbN9m+4wUbQPQXHl3/osL3a124bumLWJXpMp7CrYnJf2hpAsj4oTt2yW9XdKbJX0qIm61/VlJV0u6oer2AWiufkthD9IbadoidkVKlWg+TdKY7dMkvUzSEUnbJN2Rvb5X0o40TQNQF/vmFzQ9s1+b99yj6Zn92je/0Pf8opbCbtoidkWqvKcQEQu2/1LSdyWdkPQ1SXOSjkfEc9lphyX1/K9oe5ekXZK0cePG8hsMIIlhx/WLyE8UtYlPE1XeU7B9lqQrJG2WtF7SmZIuW+3vR8SNETEVEVMTExMltRJAainH9avYfKeuUsw+eoOk/4qIY5Jk+05J05LGbZ+W9RY2SOrfTwQw0lKP6zdpEbsipcgpfFfSxbZfZtuSLpX0kKQDkq7Mztkp6a4EbQNQE20e10+p8qAQEQfVSSjfL+lQ1oYbJV0r6Y9sPybpHEk3V902APXRtM1pRkWS4rWIuF7S9csOPy7pogTNAVBDTducZlRQ0Qygtto6rp8SC+IBALoICgCALoICAKCLoAAA6CIoAAC6mH0EQFJ7N5XBCxEUALR6Uxm8EEEBaJmUG9vTG6k/ggLQInk9guUBYVGRi8/RG2kGEs1Ai/Tb2L6XIhefa/MWl01CUABaZKWN7ZcqevG51EthY3UICkCLFLWxfZGfzVLY9UJOAWiRoja2L/qzUR8EBaBFUi5HzVLYzeCISN2GoU1NTcXs7GzqZgBAo9iei4ipXq/RUwDQ1zC1BdQjNBdBAUCuYWoLqEdoNmYfAcg1TG0B9QjNRlAAkGuY2gLqEZqNoAAg1zC1BdQjNBtBAWiwffMLmp7Zr8177tH0zH7tm18o9P13b98ycKXzML+D+iDRDDRUFQndYWoLqEdoNuoUgIaantmvhR7j9JPjY/rnPdtK/3ymnTYXdQrACEqZ0GXa6egipwA01EoJ3TLzDUw7HV0EBaCh+iV0F+/kF46fUOj5O/miAgPTTkcXw0dAA/Qbv+91fHpmf6nba64fH+uZz2DaafMRFICaW2n8vtcf+bLv5FkGe3QxfATU3DDj92UXkO3YOln6pjxIg54CUHPD3PVXcSdf9qY8SIOeAlBzw9z1cyePYdFTAGpu2Lt+7uQxjCRBwfa4pJskvUpSSPo9SY9Iuk3SJklPSLoqIn6Yon1AKoPOMgKKlqqn8BlJ/xARV9o+Q9LLJH1Y0n0RMWN7j6Q9kq5N1D6gcsPMMgKKVnlOwfbPS/otSTdLUkT8NCKOS7pC0t7stL2SdlTdNiAlqoRRBykSzZslHZP0edvztm+yfaaktRFxJDvnaUlre/2y7V22Z23PHjt2rKImA+WjShh1kCIonCbptZJuiIitkv5XnaGirugs3dpz+daIuDEipiJiamJiovTGAlVhcxrUQYqgcFjS4Yg4mD2/Q50g8YztdZKU/TyaoG1AMmxOgzqoPChExNOSnrK9+E2/VNJDku6WtDM7tlPSXVW3DUiJ2gLUQarZR++T9MVs5tHjkn5XnQB1u+2rJT0p6apEbQNKlzf1lFlGSC1JUIiIByT12vXn0oqbAlSODWpQZ1Q0AyXq1SPoN/WUoIDUCApASfJ6BMsDwiKmnqIOWBAPKElej2CN3fN8pp6iDggKQEny7vxPRjD1FLVFUABKknfnvzjVlKmnqCNyCkBJ+i15zdRT1BVBASgJS16jiQgKQInoEaBpyCkAALoICgCArhWDgu332T6risYAANJaTU9hraRv2r7d9mV2TuUNAKDxVgwKEfEnks5XZ/vM90h61Paf2/6lktsGAKjYqnIK2U5oT2f/npN0lqQ7bP9FiW0DAFRsxSmptq+R9G5J35d0k6TdEfGs7ZdIelTSh8ptIlCtvL0OgDZYTZ3C2ZLeFhFPLj0YET+zfXk5zQLSYK8DtN2KQSEiru/z2sPFNgdIa5i9DuhZYJRQ0Qwskbeyad5xehYYNRSvAUvkrWyad7xfzwJoIoICsMTu7VsG2utg0J4FUHcEBWCJHVsnB9rrYNCeBVB35BSAZQZZ2bTfnglAExEUgFPAngkYNQQF4BSxZwJGCTkFAEAXPQW0FkVnwIsRFNBKFJ0BvREU0EosZwH0RlBAK7GcBdAbiWa0EstZAL0RFNBKLGcB9EZQQCuxnAXQGzkFtBbLWQAvliwo2F4jaVbSQkRcbnuzpFslnSNpTtK7IuKnqdqH0VHErCGWs0BbpOwpXCPpYUk/lz3/uKRPRcSttj8r6WpJN6RqHEZDkbOGWM4CbZAkp2B7g6S3SLope25J2yTdkZ2yV9KOFG3DaGHWEDCYVInmT0v6kKSfZc/PkXQ8Ip7Lnh+W1POWzPYu27O2Z48dO1Z6Q9FszBoCBlN5ULB9uaSjETE3zO9HxI0RMRURUxMTEwW3DqOGWUPAYFL0FKYlvdX2E+oklrdJ+oykcduLOY4NkhYStA0l2De/oOmZ/dq85x5Nz+zXvvnq/tMOWo8AtF3lQSEirouIDRGxSdLbJe2PiHdKOiDpyuy0nZLuqrptKN5ionfh+AmFnk/0VhUYBq1HANquTnUK10q61fZHJM1Lujlxe1CAYRaek4pdfI5ZQ8DqJQ0KEfF1SV/PHj8u6aKU7UHxhkn0svgckE6degpouF539+vHx7TQIwD0S/QO27sAcOpY+wiFyMsdXHLBxMCJXqaRAukQFFCIvLv7A985NnCil2mkQDoMH6EQ/e7uB030svgckA49BRSiyLt7ppEC6dBTQCGKvrtnGimQBkEBhUi9tHSRdQ1AmxEUUJhUd/fUNQDFISigUXr1CKhrAIpDUEBj5PUIlgeERdQ1AINj9hEaI69HsMbueT51DcDgCApojLw7/5MRLI8NFISggMbIu/NfrGOgrgE4deQU0Bj9aiGoawCKQVBAUoPUF6SuhQDagKCAZIapL6BHAJSLnAKS6VdfACANegrIHcIpe+kI9k0A6oeg0HJ5QzizT/5AX55bKHXpiGF2ZQNQLoaPWi5vCOeWg0+VPrSze/sW6guAmqGn0HL9CsIGOX8YzCYC6oeg0HJ5Qzhr7J6BoeihHWYTAfXC8FHL5Q3hvOPXz2NoB2ghegotkTeTqN8QztQrz2ZoB2gZR87YcRNMTU3F7Oxs6mbU3vIZRlLnrp/1gYB2sj0XEVO9XmP4qAUoEgOwWgSFFqBIDMBqkVNoqEGqjSkSA7Ba9BQaaDFHsHD8hELPVxvvm1/oeT5FYgBWi55CA62UI8jrQTCTCMBKCAoNlJcLWL6R/fL1iggCAFbC8FED5eUC1tiVzDLaN7+g6Zn92rznHk3P7M8dtgLQPASFBsrLEVSxXtGg+QwAzVJ5ULB9nu0Dth+y/W3b12THz7Z9r+1Hs59nVd22ptixdbLnRvWTOT2IImcZUfMAjLYUOYXnJH0wIu63/QpJc7bvlfQeSfdFxIztPZL2SLo2QfsaIS9HkLexfVGoeQBGW+U9hYg4EhH3Z49/LOlhSZOSrpC0Nzttr6QdVbet6fJ6EEUmmPN6HdQ8AKMh6ewj25skbZV0UNLaiDiSvfS0pLU5v7NL0i5J2rhxYwWtbJayZxnt3r6l9N4IgHSSJZptv1zSlyW9PyJ+tPS16KzS1zNrGhE3RsRURExNTExU0FIsVUVvBEA6SXoKtk9XJyB8MSLuzA4/Y3tdRByxvU7S0RRtw8qoeQBGV4rZR5Z0s6SHI+KTS166W9LO7PFOSXdV3TYAaLsUPYVpSe+SdMj2A9mxD0uakXS77aslPSnpqgRtS2aQBe4AoCyVB4WI+CdJznn50irbUhfLN8FZvjwFAFSFiuYaoCAMQF0QFGqAgjAAdcEqqaegqDwAm+AAqAt6CkMqcmE4NsEBUBcEhSEVmQegIAxAXTB8NKSi8wAUhAGoA4LCkPrlAag5ANBUDB8NKS8PcMkFE2xCA6Cx6CksMcgd/uLx5ef3yzUM01ug1wGgSgSFzDBVxb3yAB+47YGe5w6Ta6DSGUDVGD7KFDWbqMhNaKh0BlA1gkKmqNlERdYcUOkMoGoEhUxRd/hF1hyw9SWAqpFTyBS5zWRRNQdsfQmgagSFTN5sopQJ3Tq2CcBoc2c75GaampqK2dnZ0j+nqGmhTC8FUAe25yJiqtdr9BRWUNS0UKaXAmgCEs0rKGpaKNNLATQBQWEFRU0LZXopgCYgKKygqGmhTC8F0AQEhRUUVYzGRjoAmoBE8wqKmhbK9FIATcCUVABoGaakLkGtAADka1VQoFYAAPprVaKZWgEA6K9VQYFaAQDor1VBgVoBAOivVUGBWgEA6K9ViWZqBQCgv1YFBam4DXAAYBS1avgIANAfQQEA0EVQAAB0ERQAAF0EBQBAV6NXSbV9TNKTK5x2rqTvV9CcuuG626Wt1y2199pP5bpfGRETvV5odFBYDduzeUvEjjKuu13aet1Se6+9rOtm+AgA0EVQAAB0tSEo3Ji6AYlw3e3S1uuW2nvtpVz3yOcUAACr14aeAgBglQgKAICukQ4Kti+z/Yjtx2zvSd2estj+nO2jth9ccuxs2/fafjT7eVbKNpbB9nm2D9h+yPa3bV+THR/pa7f9Utv/bvs/suv+0+z4ZtsHs+/7bbbPSN3WMtheY3ve9t9lz0f+um0/YfuQ7Qdsz2bHSvmej2xQsL1G0l9LepOkCyW9w/aFaVtVmi9IumzZsT2S7ouI8yXdlz0fNc9J+mBEXCjpYknvzf4bj/q1/0TStoj4VUmvkXSZ7YslfVzSpyLilyX9UNLV6ZpYqmskPbzkeVuu+5KIeM2S2oRSvucjGxQkXSTpsYh4PCJ+KulWSVckblMpIuIbkn6w7PAVkvZmj/dK2lFlm6oQEUci4v7s8Y/V+UMxqRG/9uj4n+zp6dm/kLRN0h3Z8ZG7bkmyvUHSWyTdlD23WnDdOUr5no9yUJiU9NSS54ezY22xNiKOZI+flrQ2ZWPKZnuTpK2SDqoF154NoTwg6aikeyX9p6TjEfFcdsqoft8/LelDkn6WPT9H7bjukPQ123O2d2XHSvmet27ntTaKiLA9snOPbb9c0pclvT8iftS5eewY1WuPiJOSXmN7XNJXJF2QtkXls325pKMRMWf79YmbU7XXRcSC7V+QdK/t7yx9scjv+Sj3FBYknbfk+YbsWFs8Y3udJGU/jyZuTylsn65OQPhiRNyZHW7FtUtSRByXdEDSb0gat714ozeK3/dpSW+1/YQ6w8HbJH1Go3/dioiF7OdRdW4CLlJJ3/NRDgrflHR+NjPhDElvl3R34jZV6W5JO7PHOyXdlbAtpcjGk2+W9HBEfHLJSyN97bYnsh6CbI9JeqM6+ZQDkq7MThu5646I6yJiQ0RsUuf/5/0R8U6N+HXbPtP2KxYfS/ptSQ+qpO/5SFc0236zOmOQayR9LiI+mrZF5bB9i6TXq7OU7jOSrpe0T9Ltkjaqs7z4VRGxPBndaLZfJ+kfJR3S82PMH1YnrzCy1277V9RJLK5R58bu9oj4M9u/qM4d9NmS5iX9TkT8JF1Ly5MNH/1xRFw+6tedXd9XsqenSfpSRHzU9jkq4Xs+0kEBADCYUR4+AgAMiKAAAOgiKAAAuggKAIAuggIAoIugAADoIigAALoICkCBbP+a7W9lex6cme138KrU7QJWi+I1oGC2PyLppZLGJB2OiI8lbhKwagQFoGDZWlvflPR/kn4zW9EUaASGj4DinSPp5ZJeoU6PAWgMegpAwWzfrc4CbZslrYuIP0jcJGDV2GQHKJDtd0t6NiK+lO0T/i+2t0XE/tRtA1aDngIAoIucAgCgi6AAAOgiKAAAuggKAIAuggIAoIugAADoIigAALr+HyoVNIIQgWfCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X.numpy(), y.numpy())\n",
    "plt.ylabel('y')\n",
    "plt.xlabel('x');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that when we created tensor $X$, we did <em>not</em> pass <tt>requires_grad=True</tt>. This means that $y$ doesn't have a gradient function, and <tt>y.backward()</tt> won't work. Since PyTorch is not tracking operations, it doesn't know the relationship between $X$ and $y$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple linear model\n",
    "As a quick demonstration we'll show how the built-in <tt>nn.Linear()</tt> model preselects weight and bias values at random."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter containing:\n",
      "tensor([[0.1060]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([0.9638], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(59)\n",
    "\n",
    "model = nn.Linear(in_features=1, out_features=1)\n",
    "print(model.weight)\n",
    "print(model.bias)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Without seeing any data, the model sets a random weight of 0.1060 and a bias of 0.9638."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model classes\n",
    "PyTorch lets us define models as object classes that can store multiple model layers. In upcoming sections we'll set up several neural network layers, and determine how each layer should perform its forward pass to the next layer. For now, though, we only need a single <tt>linear</tt> layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Model(nn.Module):\n",
    "    def __init__(self, in_features, out_features):\n",
    "        super().__init__()\n",
    "        self.linear = nn.Linear(in_features, out_features)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        y_pred = self.linear(x)\n",
    "        return y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><strong>NOTE:</strong> The \"Linear\" model layer used here doesn't really refer to linear regression. Instead, it describes the type of neural network layer employed. Linear layers are also called \"fully connected\" or \"dense\" layers. Going forward our models may contain linear layers, convolutional layers, and more.</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When <tt>Model</tt> is instantiated, we need to pass in the size (dimensions) of the incoming and outgoing features. For our purposes we'll use (1,1).<br>As above, we can see the initial hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model(\n",
      "  (linear): Linear(in_features=1, out_features=1, bias=True)\n",
      ")\n",
      "Weight: 0.10597813129425049\n",
      "Bias:   0.9637961387634277\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(59)\n",
    "model = Model(1, 1)\n",
    "print(model)\n",
    "print('Weight:', model.linear.weight.item())\n",
    "print('Bias:  ', model.linear.bias.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As models become more complex, it may be better to iterate over all the model parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linear.weight \t 0.10597813129425049\n",
      "linear.bias \t 0.9637961387634277\n"
     ]
    }
   ],
   "source": [
    "for name, param in model.named_parameters():\n",
    "    print(name, '\\t', param.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><strong>NOTE:</strong> In the above example we had our Model class accept arguments for the number of input and output features.<br>For simplicity we can hardcode them into the Model:\n",
    "         \n",
    "<tt><font color=black>\n",
    "class Model(torch.nn.Module):<br>\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;def \\_\\_init\\_\\_(self):<br>\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;super().\\_\\_init\\_\\_()<br>\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.linear = Linear(1,1)<br><br>\n",
    "model = Model()\n",
    "</font></tt><br><br>\n",
    "\n",
    "Alternatively we can use default arguments:\n",
    "\n",
    "<tt><font color=black>\n",
    "class Model(torch.nn.Module):<br>\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;def \\_\\_init\\_\\_(self, in_dim=1, out_dim=1):<br>\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;super().\\_\\_init\\_\\_()<br>\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.linear = Linear(in_dim,out_dim)<br><br>\n",
    "model = Model()<br>\n",
    "<em>\\# or</em><br>\n",
    "model = Model(i,o)</font></tt>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's see the result when we pass a tensor into the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1.1758], grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([2.0])\n",
    "print(model.forward(x))   # equivalent to print(model(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which is confirmed with $f(x) = (0.1060)(2.0)+(0.9638) = 1.1758$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the initial model\n",
    "We can plot the untrained model against our dataset to get an idea of our starting point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1. 50.]\n"
     ]
    }
   ],
   "source": [
    "x1 = np.array([X.min(),X.max()])\n",
    "print(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial weight: 0.10597813, Initial bias: 0.96379614\n",
      "\n",
      "[1.0697743 6.2627025]\n"
     ]
    }
   ],
   "source": [
    "w1,b1 = model.linear.weight.item(), model.linear.bias.item()\n",
    "print(f'Initial weight: {w1:.8f}, Initial bias: {b1:.8f}')\n",
    "print()\n",
    "\n",
    "y1 = x1*w1 + b1\n",
    "print(y1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAarElEQVR4nO3de5CldXng8e/jMEoDxubSy8LgOLORwvWyQtJrzE4qq6ABE0ooopSuq6PBZXVzIa5LBCsp3V0Ng6aibGWjS4A4boxIEIFSN4QwWObK0sOwQUQWQkBouQyRiboZYWZ49o/z9qGnOae7z5n3ct5zvp+qru7zvufyvMPh97y/e2QmkiQBPKfpACRJo8OkIEnqMilIkrpMCpKkLpOCJKnLpCBJ6jIpaCJFxP+KiM3LnP90RPzGKt/raxHx7vKiK/czIyIj4sVVx6TxYFLQ2IiI+yPidat5bma+ITO3Fq97Z0T8+ZLz78nM/1pCTB8uCuXzlhw/rzj+4QP9DKlMJgWpev8XeMeSY5uL49JIMSloLC3c/UfEb0XEExHxdxHxhkXnvxYR746Ifw58GvjJiPhBROwqzn8mIj5S/H14RHw5InYW7/XliDhugHBuBQ6JiJcV7/cy4ODi+OKY/11E3BsR342I6yPi2EXnXh8R34qIf4iI3wFiyWt/ISLuKuK7ISJeNMi/l7TApKBx9hPA3cBRwMeAyyNiv8I0M+8C3gP8VWYelpnTPd7nOcDvAy8C1gO7gd8ZMJb/yTO1hc3F466IOBm4CDgbOAZ4ALiyOHcUcA3w68W1/C2wadFrzwA+CJwFzAB/Bnx+wPgkwKSg8fZAZv5eZu4DttIpbI8e9E0y8+8z84uZ+Y+Z+X3go8C/HvBt/gB4a0SsBd5SPF7sbcAVmXlbZj4JXEin9rIB+Fngzsy8OjP3AJ8EHln02vcAF2XmXZm5F/hN4ERrCxqGSUHjrFtwZuY/Fn8eNuibRMQhEfE/IuKBiPge8HVgOiLWrPY9MvPbwL10Cux7MvPBJU85lk7tYOH5PwD+HlhXnHtw0blc/JhODeaSiNhVNH99l07z0rrVX6XUYVKQYKWlgt8PnAD8RGb+CPDTxfHo/5KePlu812d7nPsOncK988YRhwJHAvPAw8ALF52LxY/pJIh/n5nTi36mMvMvB4xPMilIwKPAcRHx3D7nn0+nH2FXRBwBfGjIz/kC8DPAVT3OfR54V0ScGBHPo1OjuCUz7we+ArwsIs6KiIOAXwH+6aLXfhq4cFFH9gsi4s1DxqgJZ1KQYBtwJ/BIRDze4/wngSngceCvgT8e5kMyc3dm/mlm7u5x7k+B3wC+SKdm8KN0+h7IzMeBNwNb6DQpHQ/8xaLXfgm4GLiyaN76BvAGpCGEm+xIkhZYU5AkdZkUJEldJgVJUpdJQZLUdVDTARyIo446Kjds2NB0GJLUKtu3b388M2d6nWt1UtiwYQNzc3NNhyFJrRIRD/Q7Z/ORJKnLpCBJ6jIpSJK6KksKEXFFRDwWEd9YdOyIiLgxIu4pfh9eHI+I+G/FBiN/ExE/VlVckqT+qqwpfAY4bcmxC4CbMvN44KbiMXTWaTm++DkX+FSFcUmS+qhs9FFmfr3YIGSxM4DXFH9vBb4GfKA4/tlinfi/jojpiDgmMx+uKj5JKtu1O+b5+A13851duzl2eorzTz2BM08qd1uLqj+j7iGpRy8q6B/hmV2w1rH/piEPFceelRQi4lw6tQnWr19fXaSSNIBrd8xz4TV3sHvPPgDmd+3mwmvuACit0K7jMxrraC5qBQMv0ZqZl2bmbGbOzsz0nHshSbX7+A13dwvrBbv37OPjN9zdqs+oOyk8GhHHABS/HyuOz7P/TlLHFcckqRW+s+tZ22Qse3xUP6PupHA9sLn4ezNw3aLj7yhGIb0a+Af7EyS1ybHTUwMdH9XPqHJI6ueBvwJOiIiHIuIcOjtHvT4i7gFeVzwG+CpwH52NzX8P+A9VxSVJVTj/1BOYWrtmv2NTa9dw/qkntOozqhx99NY+p07p8dwEfrGqWCSpagsdvVWODKrjM1q9Hefs7Gy6IJ4kDSYitmfmbK9zLnMhSeoyKUiSulq9n4IktVkdM6AHZVKQpAbUMTt5GDYfSVID6pidPAyTgiQ1oI7ZycOw+UiSGnDs9BTzPRLAsdNTjfY1WFOQpAb0m5382pfMcOE1dzC/azfJM30N1+6oZzk4k4IkNeDMk9Zx0VmvYN30FAGsm57iorNewc3f2tloX4PNR5LUkDNPWvesZqH3feH2ns+tq6/BpCBJA6qyzX+5voY62HwkSQNYmF9QVZt/HSuhLseagqRWKutufdD3WW5+QRm1hTpWQl2OSUFS65Q1G3iY96ljfkGvvoa62HwkqXXKmg08zPvUsftZk0wKklqnrLv1Yd6n6Tb/qtl8JKl1yhqhM8z7DNPmP4qrofZjUpDUOuefesJ+fQEw3N36sO8zSJv/qK6G2o/NR5Jap99s4EEL2bLeZzmjuhpqP9YUJLVSWSN0qh7pM6qrofZjTUGSKtS20UomBUmqUNtGK9l8JEkVanqG8qBMCpIa16Yhm8NocobyoEwKkhrVtiGb486kIKlRVS8wt2DcayNlMSlIalQdQzatjayeSUFSo8reVKZXjaCu2sg4cEiqpEaVOWSz3wY4vZIOjO4EsiZZU5DUqDKHbParEayJYF/ms56/XG1kUvsgGkkKEfE+4N1AAncA7wKOAa4EjgS2A2/PzKeaiE9Svcoastnvzn9fJlNr16x64btJ7oOovfkoItYBvwLMZubLgTXAW4CLgU9k5ouBJ4Bz6o5NUrv1u/NfWOhutQvftW0RuzI11Xx0EDAVEXuAQ4CHgZOBf1Oc3wp8GPhUI9FJaqXllsIepDbStkXsylR7UsjM+Yj4LeDbwG7gT+g0F+3KzL3F0x4CxruOJmlFg7brl9U/UfaIqDapPSlExOHAGcBGYBfwR8BpA7z+XOBcgPXr11cQoaRRMGy7fhn9E2Vt4tNGTQxJfR3wd5m5MzP3ANcAm4DpiFhIUscB871enJmXZuZsZs7OzMzUE7Gk2jXZrl/H5jujqok+hW8Dr46IQ+g0H50CzAE3A2+iMwJpM3BdA7FJGhFNt+u3aRG7MtVeU8jMW4CrgdvoDEd9DnAp8AHgP0bEvXSGpV5ed2ySRkfbNqcZF43MaM7MD2XmSzLz5Zn59sx8MjPvy8xXZeaLM/PNmflkE7FJGg1t25xmXDijWdJIatvmNOPCpCBpZE1qu36TXBBPktRlUpAkdZkUJEldJgVJUpcdzZKAyd0/QPszKUia6P0DtD+TgjRhmtzD2NrI6DMpSBOkX41gaUJYUOY6Q9ZG2sGOZmmCLLeHcS9lrjM0ybuZtYlJQZogK+1hvFjZ6ww1veqpVsekIE2QsvYwLvOzXfV0tNinIE2QsvYwLvuzNTpMCtIEaXLlUVc9bYfIzKZjGNrs7GzOzc01HYYktUpEbM/M2V7nrClIWtYwcwucj9BeJgVJfQ0zt8D5CO3m6CNJfQ0zt8D5CO1mUpDU1zBzC5yP0G42H0ktVnXb/bHTU8z3KMyXm1swzGs0OqwpSC210HY/v2s3yTNt99fumC/tM84/9YSBZzoP8xqNDmsKUkvVsbLpcnML+tVSnI/QbiYFqaXqarvvNdN5pRFGVc+OVnVsPpJaaqW1hK7dMc+mLdvYeMFX2LRlW6nNSo4wGl8mBamllmu7r7q/wRFG48vmI6kFlhtl1Ov4pi3bKu1vcITR+DIpSCNuNe33S1V9J++Kp+PL5iNpxA3Tfl/13gVnnrSu8v0X1AxrCtKIG+auv447eUcYjSdrCtKIG+au3zt5DcuagjTihr3r905ew2gkKUTENHAZ8HIggV8A7ga+AGwA7gfOzswnmohPasqgo4yksjVVU7gE+OPMfFNEPBc4BPggcFNmbomIC4ALgA80FJ9Uu2FGGUllq71PISJeAPw0cDlAZj6VmbuAM4CtxdO2AmfWHZvUJGcJaxQ00dG8EdgJ/H5E7IiIyyLiUODozHy4eM4jwNG9XhwR50bEXETM7dy5s6aQpeo5S1ijoImkcBDwY8CnMvMk4P/RaSrqysyk09fwLJl5aWbOZubszMxM5cFKdal6boG0Gk0khYeAhzLzluLx1XSSxKMRcQxA8fuxBmKTGuM+BBoFtXc0Z+YjEfFgRJyQmXcDpwDfLH42A1uK39fVHZtUF0cZaVQ1Nfrol4HPFSOP7gPeRafWclVEnAM8AJzdUGxSpRxlpFHWSFLIzNuB2R6nTqk5FKlSvWoEdeyYJg3LGc1SRfrVCJYmhAWOMtIocO0jqSL9agRrIno+31FGGgUmBaki/e7892U6ykgjy6QgVaTfnf/CiqWuYKpRZJ+CVJHlVjd1lJFGlUlBqojzDtRGJgWpQtYI1Db2KUiSukwKkqQuk4IkqcukIEnqMilIkrpMCpKkrhWHpEbELwN/kJlP1BCP1Ljl9jqQxt1qagpHA7dGxFURcVpEn9W8pDGwsLLp/K7dJM+sbHrtjvmmQ5NqsWJSyMxfB44HLgfeCdwTEb8ZET9acWxS7Zbb66Cfa3fMs2nLNjZe8BU2bdlmAlGrrapPITMTeKT42QscDlwdER+rMDapdv1WNu133JqFxs2KSSEizouI7cDHgL8AXpGZ7wV+HPj5iuOTatVvZdN+x4epWUijbDU1hSOAszLz1Mz8o8zcA5CZTwOnVxqdVLPzTz1hoL0OBq1ZSKNuxdFHmfmhZc7dVW44UrMGXdn02Okp5nskAHdRU1u5Sqq0xCArmy63Z4LURiYF6QC4Z4LGjUlBOkDumaBxYlLQxHLmsvRsJgVNpIX5BQt9AQvzCwATgyaaSUETabn5Bf2SgjULTQKTgibSsDOXrVlo3Ll0tiaSM5el3kwKmkjOXJZ6MyloIp150jouOusVrJueIoB101NcdNYrlp25PMhxqa3sU9DEcuay9GyNJYWIWAPMAfOZeXpEbASuBI4EtgNvz8ynmopP46OMUUPOXNakaLKmcB5wF/AjxeOLgU9k5pUR8WngHOBTTQWn8VDmqCFnLmsSNNKnEBHHAT8HXFY8DuBk4OriKVuBM5uITePFUUPSYJrqaP4k8GvA08XjI4Fdmbm3ePwQ0POWLCLOjYi5iJjbuXNn5YGq3Rw1JA2m9qQQEacDj2Xm9mFen5mXZuZsZs7OzMyUHJ2q0OQexo4akgbTRE1hE/DGiLifTsfyycAlwHRELPRxHAe4ye0YaHoP40HnI0iTrvakkJkXZuZxmbkBeAuwLTPfBtwMvKl42mbgurpjU/mGbdMvq3Yx6HwEadKN0jyFDwBXRsRHgB3A5Q3HoxIM06Zf9jpDjhqSVq/RpJCZXwO+Vvx9H/CqJuPRgek1H2CYPYyHWcFUUjlc5kKl6Nd38NqXzAzcpu+IIak5JgWVot/d/c3f2jlwm74jhqTmjFKfglpsubv7Qdv0XWdIao41BZWizLt7RwxJzbGmoFKUfXfviCGpGSYFlaLpVUTdP1kqh0lBpWnq7t79k6XymBTUKr1qBM5rkMpjUlBr9KsRLE0IC5zXIA3O0UdqjX41gjURPZ/vvAZpcCYFtUa/O/99ma6EKpXEpKDW6HfnvzCPwXkN0oGzT0GtsdxcCOc1SOUwKag1mp4LIU0Ck4IaNeikM2sEUrVMCmqMk86k0WNSUN+79aqXjnDSmTR6TAoTrt/d+twD3+WL2+crvYt3Mx1p9DgkdcL1u1v//C0P9r2LL4ub6Uijx6Qw4ZabEDbI84dx/qknOOlMGjEmhQnX7668jqUj3ExHGj32KUy4fhPCfv7H1+3Xp7BwvOy7eIeYSqPFpDAh+o0kWm5C2OyLjnCimDRhIvu0HbfB7Oxszs3NNR3GyFs6wgg6d/021UiTKSK2Z+Zsr3P2KUyA5eYDSNJiJoUJ4HwASatln0JLDTLb+NjpKeZ7JADnA0hayppCCy30Eczv2k3yzGzja3fM93y+8wEkrZY1hRZaqY+gXw3CkUSSVmJSaKF+fQFLN7Jful6RSUDSSmw+aqHlZiHXMcro2h3zbNqyjY0XfIVNW7b1bbaS1D4mhRbq10dQx3pFg/ZnSGqX2pNCRLwwIm6OiG9GxJ0RcV5x/IiIuDEi7il+H153bG3Rb82gdTWsOuqcB2m8NdGnsBd4f2beFhHPB7ZHxI3AO4GbMnNLRFwAXAB8oIH4WqFfH0G/je3L4pwHabzVXlPIzIcz87bi7+8DdwHrgDOArcXTtgJn1h1b29Wx6qh7IEjjrdHRRxGxATgJuAU4OjMfLk49Ahzd5zXnAucCrF+/voYo26XqUUb9VlV1zoM0HhrraI6Iw4AvAr+amd9bfC47q/T17DXNzEszczYzZ2dmZmqIVIu5B4I03hqpKUTEWjoJ4XOZeU1x+NGIOCYzH46IY4DHmohNK3POgzS+ak8KERHA5cBdmfnbi05dD2wGthS/r6s7tiYNspaRJFWliZrCJuDtwB0RcXtx7IN0ksFVEXEO8ABwdgOxNWLpfgdLZyJLUl1qTwqZ+edA7w2A4ZQ6YxkVy439NylIqpMzmkeAY/8ljQoXxDsAZfUDuN+BpFFhTWFIZa4B5H4HkkaFSWFIZa4B5Nh/SaPC5qMhld0P4Nh/SaPApDCk5foBnHMgqa1sPhpSv36A175kxv0GJLWWNYVFBrnD77fvcdlzDqx1SKqTSaEwzKziXv0A7/vC7T2fO0xfgzOdJdXN5qNCWaOJytxvwF3OJNXNpFAoazRRmXMOnOksqW4mhUJZd/hlzjlwlzNJdbNPoVDmjmJlzTlwlzNJdTMpFPqNJjrzpHWljQAa9H2Wi0mSqhCdnS/baXZ2Nufm5ir9jKUjgKBztz5ok1BZ7yNJByoitmfmbK9z9imsoKwRQI4kktQGJoUVlDUCyJFEktrApLCCskYAOZJIUhuYFFZQ1rwD90yQ1AaOPlpBWSOAHEkkqQ0cfSRJo+Tpp+GHP+z87N797J+F4698JWzcONRHLDf6aOJqCq46KmnVFgropQVyv4J6pWOree6TT64utt/9XXjve0u/5IlKCq46KrXY0gK6joJ6tQV0LwcdBFNTz/wcfPD+j1/wgmcf6/fcXsfWry/v33Zx2JW864gqe68DaWI9/XQ1d8nLHX/qqeHjXVpALy1kp6cHK5BX89yD2lm8tjPqITlXQGNpcQFdV0F9IAX02rXLF7KLC+hhC+Slx1paQDdhov6llttXWSrFvn3VFcr9Xr9nz/Dxrl27fIG6tIA+0LtpC+iRN1H/dVx1dMIsFNBV3j0vPVZWAd2rkD3iiHKbN6amYM2alePSRJmopOBcgQYtLaDrKKjLLKCXFqgLBXQZzRsLxy2gNQImKilAeXsdtNrevdV1BvY7tnfv8PE+97n9C9RDDtm/gC6joLaA1gSbuKQwchYK6DLbmFd6bhkFdK8C9dBD4aijyrtztoCWamdSWGzPnuo6A/s9t8wCenFhulBAH2jH4OJjFtDS2BuppBARpwGXAGuAyzJzSyUfdMUVcPHFzy6o9+1b+bX9PO95/QvZww6DmZnymjcWjj3H9QwllWtkkkJErAH+O/B64CHg1oi4PjO/WfqHHXUUnHhiOc0bFtCSxsjIJAXgVcC9mXkfQERcCZwBlJ8U3vjGzo8kaT+jdHu7Dnhw0eOHimP7iYhzI2IuIuZ27txZW3CSNAlGKSmsSmZempmzmTk7MzPTdDiSNFZGKSnMAy9c9Pi44pgkqSajlBRuBY6PiI0R8VzgLcD1DcckSRNlZDqaM3NvRPwScAOdIalXZOadDYclSRNlZJICQGZ+Ffhq03FI0qQapeYjSVLDTAqSpK7IzKZjGFpE7AQeWOFpRwGP1xDOqPG6J8ukXjdM7rUfyHW/KDN7julvdVJYjYiYy8zZpuOom9c9WSb1umFyr72q67b5SJLUZVKQJHVNQlK4tOkAGuJ1T5ZJvW6Y3Guv5LrHvk9BkrR6k1BTkCStkklBktQ11kkhIk6LiLsj4t6IuKDpeKoSEVdExGMR8Y1Fx46IiBsj4p7i9+FNxliFiHhhRNwcEd+MiDsj4rzi+Fhfe0QcHBH/OyL+T3Hd/7k4vjEibim+718oFpYcOxGxJiJ2RMSXi8djf90RcX9E3BERt0fEXHGsku/52CaFRdt7vgF4KfDWiHhps1FV5jPAaUuOXQDclJnHAzcVj8fNXuD9mflS4NXALxb/jcf92p8ETs7MVwInAqdFxKuBi4FPZOaLgSeAc5oLsVLnAXctejwp1/3azDxx0dyESr7nY5sUWLS9Z2Y+BSxs7zl2MvPrwHeXHD4D2Fr8vRU4s86Y6pCZD2fmbcXf36dTUKxjzK89O35QPFxb/CRwMnB1cXzsrhsgIo4Dfg64rHgcTMB191HJ93yck8KqtvccY0dn5sPF348ARzcZTNUiYgNwEnALE3DtRRPK7cBjwI3A3wK7MnNv8ZRx/b5/Evg14Oni8ZFMxnUn8CcRsT0izi2OVfI9H6mls1WNzMyIGNuxxxFxGPBF4Fcz83udm8eOcb32zNwHnBgR08CXgJc0G1H1IuJ04LHM3B4Rr2k4nLr9VGbOR8Q/AW6MiG8tPlnm93ycawqTvr3noxFxDEDx+7GG46lERKylkxA+l5nXFIcn4toBMnMXcDPwk8B0RCzc6I3j930T8MaIuJ9Oc/DJwCWM/3WTmfPF78fo3AS8ioq+5+OcFCZ9e8/rgc3F35uB6xqMpRJFe/LlwF2Z+duLTo31tUfETFFDICKmgNfT6U+5GXhT8bSxu+7MvDAzj8vMDXT+f96WmW9jzK87Ig6NiOcv/A38DPANKvqej/WM5oj4WTptkAvbe3602YiqERGfB15DZyndR4EPAdcCVwHr6SwvfnZmLu2MbrWI+Cngz4A7eKaN+YN0+hXG9toj4l/Q6VhcQ+fG7qrM/C8R8c/o3EEfAewA/m1mPtlcpNUpmo/+U2aePu7XXVzfl4qHBwF/mJkfjYgjqeB7PtZJQZI0mHFuPpIkDcikIEnqMilIkrpMCpKkLpOCJKnLpCBJ6jIpSJK6TApSiSLiX0bE3xR7Hhxa7Hfw8qbjklbLyWtSySLiI8DBwBTwUGZe1HBI0qqZFKSSFWtt3Qr8EPhXxYqmUivYfCSV70jgMOD5dGoMUmtYU5BKFhHX01mgbSNwTGb+UsMhSavmJjtSiSLiHcCezPzDYp/wv4yIkzNzW9OxSathTUGS1GWfgiSpy6QgSeoyKUiSukwKkqQuk4IkqcukIEnqMilIkrr+P0xGLbPf3x7DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X.numpy(), y.numpy())\n",
    "plt.plot(x1,y1,'r')\n",
    "plt.title('Initial Model')\n",
    "plt.ylabel('y')\n",
    "plt.xlabel('x');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set the loss function\n",
    "We could write our own function to apply a Mean Squared Error (MSE) that follows<br>\n",
    "\n",
    "$\\begin{split}MSE &= \\frac {1} {n} \\sum_{i=1}^n {(y_i - \\hat y_i)}^2 \\\\\n",
    "&= \\frac {1} {n} \\sum_{i=1}^n {(y_i - (wx_i + b))}^2\\end{split}$<br>\n",
    "\n",
    "Fortunately PyTorch has it built in.<br>\n",
    "<em>By convention, you'll see the variable name \"criterion\" used, but feel free to use something like \"linear_loss_func\" if that's clearer.</em>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = nn.MSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set the optimization\n",
    "Here we'll use <a href='https://en.wikipedia.org/wiki/Stochastic_gradient_descent'>Stochastic Gradient Descent</a> (SGD) with an applied <a href='https://en.wikipedia.org/wiki/Learning_rate'>learning rate</a> (lr) of 0.001. Recall that the learning rate tells the optimizer how much to adjust each parameter on the next round of calculations. Too large a step and we run the risk of overshooting the minimum, causing the algorithm to diverge. Too small and it will take a long time to converge.\n",
    "\n",
    "For more complicated (multivariate) data, you might also consider passing optional <a href='https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Momentum'><tt>momentum</tt></a> and <a href='https://en.wikipedia.org/wiki/Tikhonov_regularization'><tt>weight_decay</tt></a> arguments. Momentum allows the algorithm to \"roll over\" small bumps to avoid local minima that can cause convergence too soon. Weight decay (also called an L2 penalty) applies to biases.\n",
    "\n",
    "For more information, see <a href='https://pytorch.org/docs/stable/optim.html'><strong><tt>torch.optim</tt></strong></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.SGD(model.parameters(), lr = 0.001)\n",
    "\n",
    "# You'll sometimes see this as\n",
    "# optimizer = torch.optim.SGD(model.parameters(), lr = 1e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the model\n",
    "An <em>epoch</em> is a single pass through the entire dataset. We want to pick a sufficiently large number of epochs to reach a plateau close to our known parameters of $\\mathrm {weight} = 2,\\; \\mathrm {bias} = 1$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><strong>Let's walk through the steps we're about to take:</strong><br>\n",
    "\n",
    "1. Set a reasonably large number of passes<br>\n",
    "<tt><font color=black>epochs = 50</font></tt><br>\n",
    "2. Create a list to store loss values. This will let us view our progress afterward.<br>\n",
    "<tt><font color=black>losses = []</font></tt><br>\n",
    "<tt><font color=black>for i in range(epochs):</font></tt><br>\n",
    "3. Bump \"i\" so that the printed report starts at 1<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;i+=1</font></tt><br>\n",
    "4. Create a prediction set by running \"X\" through the current model parameters<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;y_pred = model.forward(X)</font></tt><br>\n",
    "5. Calculate the loss<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;loss = criterion(y_pred, y)</font></tt><br>\n",
    "6. Add the loss value to our tracking list<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;losses.append(loss)</font></tt><br>\n",
    "7. Print the current line of results<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;print(f'epoch: {i:2} loss: {loss.item():10.8f}')</font></tt><br>\n",
    "8. Gradients accumulate with every backprop. To prevent compounding we need to reset the stored gradient for each new epoch.<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;optimizer.zero_grad()</font></tt><br>\n",
    "9. Now we can backprop<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;loss.backward()</font></tt><br>\n",
    "10. Finally, we can update the hyperparameters of our model<br>\n",
    "<tt><font color=black>&nbsp;&nbsp;&nbsp;&nbsp;optimizer.step()</font></tt>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:  1  loss: 3057.21679688  weight: 0.10597813  bias: 0.96379614\n",
      "epoch:  2  loss: 1588.53112793  weight: 3.33490038  bias: 1.06046367\n",
      "epoch:  3  loss: 830.29998779  weight: 1.01483274  bias: 0.99226284\n",
      "epoch:  4  loss: 438.85241699  weight: 2.68179965  bias: 1.04252183\n",
      "epoch:  5  loss: 236.76152039  weight: 1.48402119  bias: 1.00766504\n",
      "epoch:  6  loss: 132.42912292  weight: 2.34460592  bias: 1.03396463\n",
      "epoch:  7  loss: 78.56572723  weight: 1.72622538  bias: 1.01632178\n",
      "epoch:  8  loss: 50.75775909  weight: 2.17050409  bias: 1.03025162\n",
      "epoch:  9  loss: 36.40123367  weight: 1.85124576  bias: 1.02149546\n",
      "epoch: 10  loss: 28.98922920  weight: 2.08060074  bias: 1.02903891\n",
      "epoch: 11  loss: 25.16238213  weight: 1.91576838  bias: 1.02487016\n",
      "epoch: 12  loss: 23.18647385  weight: 2.03416562  bias: 1.02911627\n",
      "epoch: 13  loss: 22.16612625  weight: 1.94905841  bias: 1.02731562\n",
      "epoch: 14  loss: 21.63911057  weight: 2.01017213  bias: 1.02985907\n",
      "epoch: 15  loss: 21.36677170  weight: 1.96622372  bias: 1.02928054\n",
      "epoch: 16  loss: 21.22591782  weight: 1.99776423  bias: 1.03094459\n",
      "epoch: 17  loss: 21.15294647  weight: 1.97506487  bias: 1.03099668\n",
      "epoch: 18  loss: 21.11500931  weight: 1.99133754  bias: 1.03220642\n",
      "epoch: 19  loss: 21.09517670  weight: 1.97960854  bias: 1.03258383\n",
      "epoch: 20  loss: 21.08468437  weight: 1.98799884  bias: 1.03355861\n",
      "epoch: 21  loss: 21.07901382  weight: 1.98193336  bias: 1.03410351\n",
      "epoch: 22  loss: 21.07583046  weight: 1.98625445  bias: 1.03495669\n",
      "epoch: 23  loss: 21.07393837  weight: 1.98311269  bias: 1.03558779\n",
      "epoch: 24  loss: 21.07270050  weight: 1.98533309  bias: 1.03637791\n",
      "epoch: 25  loss: 21.07181931  weight: 1.98370099  bias: 1.03705311\n",
      "epoch: 26  loss: 21.07110596  weight: 1.98483658  bias: 1.03781021\n",
      "epoch: 27  loss: 21.07048607  weight: 1.98398376  bias: 1.03850794\n",
      "epoch: 28  loss: 21.06991386  weight: 1.98455977  bias: 1.03924775\n",
      "epoch: 29  loss: 21.06936836  weight: 1.98410904  bias: 1.03995669\n",
      "epoch: 30  loss: 21.06883812  weight: 1.98439610  bias: 1.04068720\n",
      "epoch: 31  loss: 21.06830788  weight: 1.98415291  bias: 1.04140162\n",
      "epoch: 32  loss: 21.06778145  weight: 1.98429084  bias: 1.04212701\n",
      "epoch: 33  loss: 21.06726074  weight: 1.98415494  bias: 1.04284394\n",
      "epoch: 34  loss: 21.06674004  weight: 1.98421574  bias: 1.04356635\n",
      "epoch: 35  loss: 21.06622505  weight: 1.98413551  bias: 1.04428422\n",
      "epoch: 36  loss: 21.06570816  weight: 1.98415649  bias: 1.04500473\n",
      "epoch: 37  loss: 21.06518745  weight: 1.98410451  bias: 1.04572272\n",
      "epoch: 38  loss: 21.06466866  weight: 1.98410523  bias: 1.04644191\n",
      "epoch: 39  loss: 21.06415749  weight: 1.98406804  bias: 1.04715967\n",
      "epoch: 40  loss: 21.06363678  weight: 1.98405814  bias: 1.04787791\n",
      "epoch: 41  loss: 21.06312561  weight: 1.98402870  bias: 1.04859519\n",
      "epoch: 42  loss: 21.06260681  weight: 1.98401320  bias: 1.04931259\n",
      "epoch: 43  loss: 21.06209564  weight: 1.98398757  bias: 1.05002928\n",
      "epoch: 44  loss: 21.06157684  weight: 1.98396957  bias: 1.05074584\n",
      "epoch: 45  loss: 21.06106949  weight: 1.98394585  bias: 1.05146194\n",
      "epoch: 46  loss: 21.06055450  weight: 1.98392630  bias: 1.05217779\n",
      "epoch: 47  loss: 21.06004333  weight: 1.98390377  bias: 1.05289316\n",
      "epoch: 48  loss: 21.05953217  weight: 1.98388338  bias: 1.05360830\n",
      "epoch: 49  loss: 21.05901337  weight: 1.98386145  bias: 1.05432308\n",
      "epoch: 50  loss: 21.05850983  weight: 1.98384094  bias: 1.05503750\n"
     ]
    }
   ],
   "source": [
    "epochs = 50\n",
    "losses = []\n",
    "\n",
    "for i in range(epochs):\n",
    "    i+=1\n",
    "    y_pred = model.forward(X)\n",
    "    loss = criterion(y_pred, y)\n",
    "    losses.append(loss)\n",
    "    print(f'epoch: {i:2}  loss: {loss.item():10.8f}  weight: {model.linear.weight.item():10.8f}  \\\n",
    "bias: {model.linear.bias.item():10.8f}') \n",
    "    optimizer.zero_grad()\n",
    "    loss.backward() \n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You may encounter the following error => \"Can't call numpy() on Tensor that requires grad\"\n",
    "losses= [ loss.detach().numpy() for loss in losses]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the loss values\n",
    "Let's see how loss changed over time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcl0lEQVR4nO3df5TddX3n8edrZu5k7gyZyQwZ0pgJZhJT2HiKAWOCBVvAAtF6BNcfC2s1tnSjK57Vrdtd8GwP1ZY92lNldVeoWFJC14qsypqjaTXSKKVWQoIRIYESQ2iShmQgIT/Ir/nx3j/uZ5KbML/n3vnO3Pt6nHPP/X4/3+/93vdXb3jN99fno4jAzMxsKDVZF2BmZpOfw8LMzIblsDAzs2E5LMzMbFgOCzMzG1Zd1gWUw8yZM2PevHlZl2FmNqVs2rTpxYhoH2hZRYbFvHnz2LhxY9ZlmJlNKZKeH2yZT0OZmdmwHBZmZjYsh4WZmQ3LYWFmZsNyWJiZ2bAcFmZmNiyHhZmZDcthUeTw8W7uWPfPbN75ctalmJlNKg6LIr19wRcfepbHnz+QdSlmZpNK2cJCUoOkDZJ+LukpSZ9O7Z2SHpW0TdI3JNWn9mlpfltaPq9oW7em9mckXVuumqc35AA4eKy7XF9hZjYllfPI4gRwVUS8AVgMLJd0KfA54I6IeB1wALgprX8TcCC135HWQ9Ii4Abg9cBy4E5JteUouLZGTJ9W57AwMztL2cIiCo6k2Vx6BXAV8M3Uvhq4Pk1fl+ZJy98qSan9/og4ERHPAduApeWquzmf49Bxh4WZWbGyXrOQVCtpM7APWAf8Eng5InrSKruAOWl6DrATIC0/CJxb3D7AZ4q/a6WkjZI2dnV1jbnmlnyOQz6yMDM7Q1nDIiJ6I2Ix0EHhaODCMn7X3RGxJCKWtLcP2MPuiDTn6zh0rGf4Fc3MqsiE3A0VES8D64E3AzMk9XeN3gHsTtO7gbkAaXkL8FJx+wCfKbmWfM7XLMzMzlLOu6HaJc1I03ngamArhdB4T1ptBfCdNL0mzZOW/31ERGq/Id0t1QksBDaUq26HhZnZq5Vz8KPZwOp051IN8EBEfFfSFuB+SX8K/Ay4J61/D/DXkrYB+yncAUVEPCXpAWAL0APcHBG95Sq6ucEXuM3Mzla2sIiIJ4CLB2jfzgB3M0XEceC9g2zrduD2Utc4kJZ8jqMne+nu7SNX62cWzczAT3C/SnPeD+aZmZ3NYXGWlhQWvn3WzOw0h8VZWnxkYWb2Kg6LszTnC5dxHBZmZqc5LM5y6jTUcT+YZ2bWz2FxFl/gNjN7NYfFWZobfIHbzOxsDouzNORqmVZX47AwMyvisBhAs7v8MDM7g8NiAC0e08LM7AwOiwG4M0EzszM5LAbQ3OChVc3MijksBlAYLc/PWZiZ9XNYDMCnoczMzuSwGEBzPsfh49309UXWpZiZTQoOiwG05HP0BRw56VNRZmbgsBhQ/1PcB4/6VJSZGTgsBtR8qjNBh4WZGTgsBuQxLczMzuSwGED/mBbuH8rMrMBhMYDTQ6v6AreZGTgsBuTTUGZmZ3JYDKCpvo4a+QK3mVm/soWFpLmS1kvaIukpSR9P7X8sabekzen19qLP3Cppm6RnJF1b1L48tW2TdEu5au5XUyN3U25mVqSujNvuAT4ZEY9Lmg5skrQuLbsjIv68eGVJi4AbgNcDrwF+KOlX0+IvA1cDu4DHJK2JiC1lrN1dfpiZFSlbWETEHmBPmj4saSswZ4iPXAfcHxEngOckbQOWpmXbImI7gKT707plDYvmhpzvhjIzSybkmoWkecDFwKOp6WOSnpC0SlJrapsD7Cz62K7UNlh7WfnIwszstLKHhaRzgG8Bn4iIQ8BdwAJgMYUjj8+X6HtWStooaWNXV9e4t9ec95gWZmb9yhoWknIUguJrEfFtgIjYGxG9EdEHfJXTp5p2A3OLPt6R2gZrP0NE3B0RSyJiSXt7+7hrLwyt6ucszMygvHdDCbgH2BoRXyhqn1202ruAJ9P0GuAGSdMkdQILgQ3AY8BCSZ2S6ilcBF9Trrr7+W4oM7PTynk31GXAB4BfSNqc2j4F3ChpMRDADuDDABHxlKQHKFy47gFujoheAEkfA74P1AKrIuKpMtYNFC5wn+zp43h3Lw252nJ/nZnZpFbOu6EeATTAorVDfOZ24PYB2tcO9blyON3lR7fDwsyqnp/gHoS7/DAzO81hMQiPaWFmdprDYhA+sjAzO81hMYjmhsLlHIeFmZnDYlAe08LM7DSHxSCafRrKzOwUh8UgcrU1NNbXujNBMzMcFkNyZ4JmZgUOiyE4LMzMChwWQ2huyPk5CzMzHBZDKnQm6LuhzMwcFkNoyXu0PDMzcFgMqTlf57AwM8NhMaSWfI7DJ3ro7YusSzEzy5TDYgjNDYUH8w77IreZVTmHxRDcmaCZWYHDYggOCzOzAofFEJrdmaCZGeCwGJKPLMzMChwWQ3BYmJkVOCyG0JwvDIDkLj/MrNo5LIaQz9WSq5WPLMys6jkshiCp0Jmgw8LMqpzDYhjuptzMrIxhIWmupPWStkh6StLHU3ubpHWSnk3vraldkr4kaZukJyRdUrStFWn9ZyWtKFfNA2l2WJiZlfXIogf4ZEQsAi4Fbpa0CLgFeCgiFgIPpXmAtwEL02slcBcUwgW4DVgGLAVu6w+YidCcz3HouJ+zMLPqVrawiIg9EfF4mj4MbAXmANcBq9Nqq4Hr0/R1wH1R8FNghqTZwLXAuojYHxEHgHXA8nLVfTZ3U25mNkHXLCTNAy4GHgVmRcSetOgFYFaangPsLPrYrtQ2WPvZ37FS0kZJG7u6ukpWe0u+zqehzKzqlT0sJJ0DfAv4REQcKl4WEQGUpP/viLg7IpZExJL29vZSbBLg1N1QhVLNzKpTWcNCUo5CUHwtIr6dmvem00uk932pfTcwt+jjHaltsPYJ0ZLP0dMXHD3ZO1FfaWY26ZTzbigB9wBbI+ILRYvWAP13NK0AvlPU/sF0V9SlwMF0uur7wDWSWtOF7WtS24RodpcfZmbUlXHblwEfAH4haXNq+xTwWeABSTcBzwPvS8vWAm8HtgFHgd8FiIj9kv4EeCyt95mI2F/Gus/Q3z/UoePdvIb8RH2tmdmkUrawiIhHAA2y+K0DrB/AzYNsaxWwqnTVjdypzgSP+sjCzKqXn+AeRv/Qqn7WwsyqmcNiGO6m3MzMYTEsh4WZmcNiWOc0pDEtHBZmVsUcFsOorRHTG/wUt5lVN4fFCLh/KDOrdg6LEWhuyHloVTOrag6LEfAASGZW7RwWI9Ccr+PQMT9nYWbVy2ExAj6yMLNq57AYAYeFmVU7h8UINDfkONbdy8mevqxLMTPLhMNiBFoaT/c8a2ZWjRwWI+AuP8ys2jksRuBUz7MOCzOrUg6LEfBoeWZW7UYUFpKaJNWk6V+V9M40vnZVaMmnzgQ9poWZVamRHlk8DDRImgP8gMJwqfeWq6jJxkcWZlbtRhoWioijwL8F7oyI9wKvL19Zk4uvWZhZtRtxWEh6M/B+4HuprbY8JU0+DblaptXVOCzMrGqNNCw+AdwKPBgRT0maD6wvW1WTUEs+x8tHHRZmVp3qRrJSRPwY+DFAutD9YkT8p3IWNtnMam5g7+HjWZdhZpaJkd4N9TeSmiU1AU8CWyT9YXlLm1w6WvPs3H806zLMzDIx0tNQiyLiEHA98LdAJ4U7ogYlaZWkfZKeLGr7Y0m7JW1Or7cXLbtV0jZJz0i6tqh9eWrbJumW0excKXW05tl14BgRkVUJZmaZGWlY5NJzFdcDayKiGxjuv5r3AssHaL8jIhan11oASYuAGyjcYbUcuFNSraRa4MvA24BFwI1p3QnX0drIiZ4+XjxyMouvNzPL1EjD4ivADqAJeFjSa4FDQ30gIh4G9o9w+9cB90fEiYh4DtgGLE2vbRGxPSJOAvendSdcR2segF0HfCrKzKrPiMIiIr4UEXMi4u1R8Dxw5Ri/82OSnkinqVpT2xxgZ9E6u1LbYO0Tbm5bY6GAA8ey+Hozs0yN9AJ3i6QvSNqYXp+ncJQxWncBC4DFwB7g82PYxmA1ruyvr6urq1SbPWXOjMKRxU4fWZhZFRrpaahVwGHgfel1CPir0X5ZROyNiN6I6AO+SuE0E8BuYG7Rqh2pbbD2gbZ9d0QsiYgl7e3toy1tWE3T6mhrqveRhZlVpZGGxYKIuC1dO9geEZ8G5o/2yyTNLpp9F4XbcAHWADdImiapE1gIbAAeAxZK6pRUT+Ei+JrRfm+p9N8RZWZWbUb0UB5wTNLlEfEIgKTLgCH/qynp68AVwExJu4DbgCskLaZwJ9UO4MMA6anwB4AtQA9wc0T0pu18DPg+he5FVkXEU6PZwVKa29rI1heGvK5vZlaRRhoWHwHuk9SS5g8AK4b6QETcOEDzPUOsfztw+wDta4G1I6yzrDpa8/xw614iAklZl2NmNmFGejfUzyPiDcBFwEURcTFwVVkrm4Q6WvOc6Omj6/CJrEsxM5tQoxopLyIOpSe5Af6gDPVMah2thdtnd/q6hZlVmfEMq1p152H8YJ6ZVavxhEXVdZLUf2ThO6LMrNoMeYFb0mEGDgUB+bJUNInl62uZeU69jyzMrOoMGRYRMX2iCpkq5rQ2+sjCzKrOeE5DVSU/mGdm1chhMUodrXl2HzhGX1/VXbIxsyrmsBilua2NnOzto+uIn7Uws+rhsBil/ttnPcSqmVUTh8Uo+fZZM6tGDotR8oN5ZlaNHBaj1JCrpX36NB9ZmFlVcViMgW+fNbNq47AYg47WRg+vamZVxWExBh2tef715WP0+lkLM6sSDosx6GjN090b7Dt8POtSzMwmhMNiDOb69lkzqzIOizHw7bNmVm0cFmPwmhn9T3H7yMLMqoPDYgwacrWcN32ajyzMrGo4LMbIz1qYWTVxWIzR3DYPgmRm1cNhMUZ+1sLMqknZwkLSKkn7JD1Z1NYmaZ2kZ9N7a2qXpC9J2ibpCUmXFH1mRVr/WUkrylXvaHW0NtLTF7xwyM9amFnlK+eRxb3A8rPabgEeioiFwENpHuBtwML0WgncBYVwAW4DlgFLgdv6AyZrp26f9bgWZlYFyhYWEfEwsP+s5uuA1Wl6NXB9Uft9UfBTYIak2cC1wLqI2B8RB4B1vDqAMuEH88ysmkz0NYtZEbEnTb8AzErTc4CdRevtSm2Dtb+KpJWSNkra2NXVVdqqBzB7RgOSw8LMqkNmF7gjIoCSXR2OiLsjYklELGlvby/VZgc1ra6WWdMb3PusmVWFiQ6Lven0Eul9X2rfDcwtWq8jtQ3WPikUnrVwWJhZ5ZvosFgD9N/RtAL4TlH7B9NdUZcCB9Ppqu8D10hqTRe2r0ltk4IfzDOzalHOW2e/DvwTcIGkXZJuAj4LXC3pWeC30jzAWmA7sA34KvBRgIjYD/wJ8Fh6fSa1TQpz2xrZc/A4Pb19WZdiZlZWdeXacETcOMiitw6wbgA3D7KdVcCqEpZWMh2teXrTsxYd6e4oM7NK5Ce4x6E/INz7rJlVOofFOHhcCzOrFg6LcZjdkvezFmZWFRwW41BfV8Ps5gaHhZlVPIfFOHW0Nvo0lJlVPIfFOHXObOKZvYfpc1flZlbBHBbj9KbONl4+2s22riNZl2JmVjYOi3Fa1tkGwKPbX8q4EjOz8nFYjFNHa57ZLQ08+tykebDczKzkHBbjJIllnW08+tx+Cg+im5lVHodFCSztPJeuwyfY8ZLvijKzyuSwKIGl6brFhud83cLMKpPDogQWtDcx85x6Ht3u6xZmVpkcFiUgiaXpuoWZWSVyWJTIss5z2f3yMT/NbWYVyWFRIqevW/jowswqj8OiRC6YNZ2WfM5hYWYVyWFRIjU14k3zfN3CzCqTw6KElnW28dyLr7Dv0PGsSzEzKymHRQktm5/6ifLRhZlVGIdFCS2a3UxTfa2vW5hZxXFYlFBdbQ1vnNfmsDCziuOwKLFlnW08s/cw+185mXUpZmYl47Aosf7xLR7b4aMLM6scmYSFpB2SfiFps6SNqa1N0jpJz6b31tQuSV+StE3SE5IuyaLmkfq1jham1dW4nygzqyhZHllcGRGLI2JJmr8FeCgiFgIPpXmAtwEL02slcNeEVzoK0+pqueT8VjbscA+0ZlY5JtNpqOuA1Wl6NXB9Uft9UfBTYIak2RnUN2JLO9vY8q+HOHS8O+tSzMxKIquwCOAHkjZJWpnaZkXEnjT9AjArTc8BdhZ9dldqO4OklZI2StrY1dVVrrpHZNn8NvoCNu04kGkdZmalklVYXB4Rl1A4xXSzpN8oXhiF8UlHNUZpRNwdEUsiYkl7e3sJSx29i+e2kquVH84zs4qRSVhExO70vg94EFgK7O0/vZTe96XVdwNziz7ekdomrXx9LRd1zPDIeWZWMSY8LCQ1SZrePw1cAzwJrAFWpNVWAN9J02uAD6a7oi4FDhadrpq0lnW28cSugxzw8xZmVgGyOLKYBTwi6efABuB7EfF3wGeBqyU9C/xWmgdYC2wHtgFfBT468SWP3rsunkNPX3DvT3ZkXYqZ2bjVTfQXRsR24A0DtL8EvHWA9gBunoDSSmrhrOlcvWgW9/5kByt/Yz5N0yb8f2ozs5KZTLfOVpyPXrGAg8e6+fqGf8m6FDOzcXFYlNHF57fy5vnn8tV/2M6Jnt6syzEzGzOHRZl99MoF7D10ggcfn9Q3cJmZDclhUWaXv24mvzanha88vJ3evlE9OmJmNmk4LMpMEv/xigU89+Ir/O2Tk/6OXzOzATksJsC1r/8V5s9s4s71v6Rwc5eZ2dTisJgAtTXiI7+5gC17DvHwsy9mXY6Z2ag5LCbI9RfPYXZLA3eu35Z1KWZmo+awmCD1dTX8/lvm8+hz+9n0vHujNbOpxWExgW5cOpfWxhx3/chHF2Y2tTgsJlBjfR0f+vVOfrh1H49ud4+0ZjZ1OCwm2Icum8f8mU38h/s28vQLh7Iux8xsRBwWE6wln2P17y0lX1/LB+/ZwM79R7MuycxsWA6LDMxta+S+31vG8e5ePrhqAy8eOZF1SWZmQ3JYZOSCX5nOqg+9iT0Hj/Ghv9rA4ePdWZdkZjYoh0WGlsxr4873X8LWPYf58F9vcs+0ZjZpOSwydtWFs/izd1/ET375Ev/5G5vd2aCZTUoevm0SePcbOzhw9CR/+r2tdB3+J/7oHYu4qGNG1mWZmZ3iI4tJ4vffMp8/e/dFbO96hXf+73/kD76xmT0Hj2VdlpkZ4LCYVN73prn86A+v4CO/uYDvPrGHK//8R3xh3T9z9GRP1qWZWZVTJXaZvWTJkti4cWPWZYzLzv1H+dzfPc13n9jDrOZp3HR5J1ddOIsF7U1Iyro8M6tAkjZFxJIBlzksJrdNz+/nf6x9+lTng+e3NXLlBe1ceeF5XDr/XBpytRlXaGaVwmFRAXYdOMqPnuli/dP7+Mdfvsjx7j7yuVre+NpWFrQ3Mb/9HDpnNtE5s4k5M/LU1Pjow8xGpyLCQtJy4ItALfCXEfHZwdatxLAodry7l59uf4n1T+/jZztfZnvXKxw5cfq6xrS6Gs5va6StqZ62pnpam+ppayy8tzbmaKyvo7G+lnx9LflcLQ25wnR9bQ25WpGrraGuVuRqahw6ZlVkqLCYErfOSqoFvgxcDewCHpO0JiK2ZFtZNhpytVxxwXlcccF5AEQEXUdOsL3rFZ578RW2dx3hX/Yf5cAr3Ty77wgHXjnJgaMnGcsjHDWCupoaamqgVqKmRtRI1KZ3qbCO6J8uhItUeEFh2am2tN3+6y6nomiITBpska/d2FCq9ddx4exm/teNF5d8u1MiLIClwLaI2A4g6X7gOqAqw+JskjhvegPnTW/g0vnnDrhOX19w+HgP+4+e5OjJHo5393LsZB/Huns5erKHYyd76e7to7s36OkrvBfm++jtg74IevuCvgj6+oLeCHr7AIKIwvIICArTpGDqz6eIKJp+9bLBDLpkahwQW0aiin8gc1vzZdnuVAmLOcDOovldwLLiFSStBFYCnH/++RNX2RRRUyNaGnO0NOayLsXMpqCKec4iIu6OiCURsaS9vT3rcszMKspUCYvdwNyi+Y7UZmZmE2CqhMVjwEJJnZLqgRuANRnXZGZWNabENYuI6JH0MeD7FG6dXRURT2VclplZ1ZgSYQEQEWuBtVnXYWZWjabKaSgzM8uQw8LMzIblsDAzs2FNmb6hRkNSF/D8ODYxE3ixROVMJd7v6uL9ri4j2e/XRsSAD6pVZFiMl6SNg3WmVcm839XF+11dxrvfPg1lZmbDcliYmdmwHBYDuzvrAjLi/a4u3u/qMq799jULMzMblo8szMxsWA4LMzMblsOiiKTlkp6RtE3SLVnXU06SVknaJ+nJorY2SeskPZveW7OssdQkzZW0XtIWSU9J+nhqr/T9bpC0QdLP035/OrV3Sno0/d6/kXp0rjiSaiX9TNJ303y17PcOSb+QtFnSxtQ25t+6wyIpGuf7bcAi4EZJi7KtqqzuBZaf1XYL8FBELAQeSvOVpAf4ZEQsAi4Fbk7/H1f6fp8AroqINwCLgeWSLgU+B9wREa8DDgA3ZVdiWX0c2Fo0Xy37DXBlRCwuer5izL91h8Vpp8b5joiTQP843xUpIh4G9p/VfB2wOk2vBq6fyJrKLSL2RMTjafowhf+AzKHy9zsi4kiazaVXAFcB30ztFbffAJI6gN8G/jLNiyrY7yGM+bfusDhtoHG+52RUS1ZmRcSeNP0CMCvLYspJ0jzgYuBRqmC/06mYzcA+YB3wS+DliOhJq1Tq7/1/Av8V6Evz51Id+w2FPwh+IGmTpJWpbcy/9SkznoVNrIgISRV5X7Wkc4BvAZ+IiEOFPzYLKnW/I6IXWCxpBvAgcGG2FZWfpHcA+yJik6QrMi4nC5dHxG5J5wHrJD1dvHC0v3UfWZzmcb5hr6TZAOl9X8b1lJykHIWg+FpEfDs1V/x+94uIl4H1wJuBGZL6/2CsxN/7ZcA7Je2gcFr5KuCLVP5+AxARu9P7Pgp/ICxlHL91h8VpHue7sL8r0vQK4DsZ1lJy6Xz1PcDWiPhC0aJK3+/2dESBpDxwNYXrNeuB96TVKm6/I+LWiOiIiHkU/j3/fUS8nwrfbwBJTZKm908D1wBPMo7fup/gLiLp7RTOcfaP8317thWVj6SvA1dQ6LZ4L3Ab8P+AB4DzKXTx/r6IOPsi+JQl6XLgH4BfcPoc9qcoXLeo5P2+iMLFzFoKfyA+EBGfkTSfwl/cbcDPgN+JiBPZVVo+6TTUf4mId1TDfqd9fDDN1gF/ExG3SzqXMf7WHRZmZjYsn4YyM7NhOSzMzGxYDgszMxuWw8LMzIblsDAzs2E5LMwmGUlX9PeQajZZOCzMzGxYDguzMZL0O2mciM2SvpI66zsi6Y40bsRDktrTuosl/VTSE5Ie7B9HQNLrJP0wjTXxuKQFafPnSPqmpKclfU3FHViZZcBhYTYGkv4N8O+AyyJiMdALvB9oAjZGxOuBH1N4Mh7gPuC/RcRFFJ4g72//GvDlNNbErwP9PYJeDHyCwtgq8yn0c2SWGfc6azY2bwXeCDyW/ujPU+iUrQ/4Rlrn/wDfltQCzIiIH6f21cD/TX33zImIBwEi4jhA2t6GiNiV5jcD84BHyr5XZoNwWJiNjYDVEXHrGY3SH5213lj70ynuq6gX/1u1jPk0lNnYPAS8J40V0D+28Wsp/Jvq79H03wOPRMRB4ICkt6T2DwA/TqP17ZJ0fdrGNEmNE7kTZiPlv1bMxiAitkj67xRGIqsBuoGbgVeApWnZPgrXNaDQHfRfpDDYDvxuav8A8BVJn0nbeO8E7obZiLnXWbMSknQkIs7Jug6zUvNpKDMzG5aPLMzMbFg+sjAzs2E5LMzMbFgOCzMzG5bDwszMhuWwMDOzYf1/79umbjMYxSEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(epochs), losses)\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('epoch');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the result\n",
    "Now we'll derive <tt>y1</tt> from the new model to plot the most recent best-fit line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current weight: 1.98381913, Current bias: 1.05575156\n",
      "\n",
      "[ 1. 50.]\n",
      "[  3.0395708 100.246704 ]\n"
     ]
    }
   ],
   "source": [
    "w1,b1 = model.linear.weight.item(), model.linear.bias.item()\n",
    "print(f'Current weight: {w1:.8f}, Current bias: {b1:.8f}')\n",
    "print()\n",
    "\n",
    "y1 = x1*w1 + b1\n",
    "print(x1)\n",
    "print(y1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqVklEQVR4nO3deZyW8/7H8dfHSEWHSJaixS5rjCwlFHLod5oTPwfHVilriKI6ynFEC622RJE1lBKdnFbKaWESrcjvKC3SoELS+vn9cd0zZ5rmnpl75r7va+6538/Ho8fMfd3XfV+f65zbfO7v9vmauyMiIgKwR9gBiIhI+aGkICIieZQUREQkj5KCiIjkUVIQEZE8SgoiIpJHSUGkAjEzN7OjSnDe+Wa2KhkxSWpRUpCUZGbXmFm2mf1qZt+Z2UQzaxJ2XABmdqOZfVTMOR9E/oCfUuD42Mjx8xMZo0g0SgqScszsHmAQ8ChwMFAHeBpoVYr32rMkxxLkK+D6fNetAZwN5CTp+iK7UVKQlGJm+wH/AG5397fdfZO7b3P3d929S+ScF82sV77X7NJVYmbLzex+M1sAbDKzoyLfztuZ2bfAtMh5bc1sqZmtN7N/mVndfO/hZnaLmS0zsw1m9pQFjgeGAmdHWjEbiridV4G/mFlG5PHVwFhga77rVDazQWa2JvJvkJlVzvd8l0hLaY2ZtS3wv1VlM3vczL41s+/NbKiZVY3tf3FJN0oKkmrOBqoQ/PEsi6uBy4DqwPbIsfOA44EWZtYK6A60BmoCM4HXC7xHS+AM4GTgSqCFuy8FbgFmu3s1d69eRAxrgCXAxZHH1wMvFTjnb8BZwKnAKUAj4AEAM7sE6AxcBBwNXFjgtX2AYyKvPQqoDfQsIh4RJQVJOTWAH9x9e7FnFm2Iu6909835jv090vLYTPCHvbe7L41c61Hg1PytBaCPu29w92+B6QR/fGP1EnC9mR0HVHf32QWe/yvwD3df5+45wEPAdZHnrgRecPdF7r4J+Hvui8zMgA5AJ3f/yd1/idzDVaWIUdJIsvpOReLlR+BAM9uzjIlhZTHH6gKDzax/vmNG8G17ReTx2nzP/QZUK0UcbwP9Ce7r5UKer5XvekR+r5XvuXkFnstVE9gbmBfkByCIPwORIqilIKlmNrAFyCrinE0EfxBzHVLIOYWVB85/bCVws7tXz/evqrvPKkGMJS497O6/AROBWyk8KawhSFC56kSOAXwHHF7guVw/AJuBE/LFv5+7lyZxSRpRUpCU4u4bCfrFnzKzLDPb28wqmdkfzaxf5LTPgEvN7AAzOwS4uxSXGgp0M7MTIBjgNrP/LeFrvwcOM7O9Snh+d+A8d19eyHOvAw+YWU0zO5Dg3l+JPPcmcKOZNTCzvYEHc1/k7juB54CBZnZQ5B5qm1mLEsYkaUpJQVKOu/cH7iEYcM0h+FZ/BzAucsrLwOfAcmAS8EYprjEW6AuMMrOfgUXAH0v48mnAYmCtmf1Qgmutcfdo6xp6AdnAAmAh8GnkGO4+kWBq7jTg68jP/O6PHJ8TuYcpwLElvAdJU6ZNdkREJJdaCiIikkdJQURE8igpiIhIHiUFERHJk9KL1w488ECvV69e2GGIiKSUefPm/eDuNQt7LqWTQr169cjOzg47DBGRlGJmK6I9p+4jERHJo6QgIiJ5lBRERCSPkoKIiORRUhARkTxKCiIikidhScHMRpjZOjNblO/YAWY2ObKv7WQz2z9y3MxsiJl9bWYLzOy0RMUlIiLRJbKl8CJwSYFjXYGp7n40MDXyGIKSxEdH/nUAnklgXCIiqeu33+D++2FF1KUGZZKwxWvuPsPM6hU43Ao4P/L7SOADgprvrYCXPKjjPcfMqpvZoe7+XaLiExGJt3HzV/PYv75kzYbN1KpelS4tjiWrYe34XWD6dDZd34Z9Vq3ggc9+ZfoFl8f9GskeUzg43x/6tcDBkd9rs+v+uKsix3ZjZh3MLNvMsnNychIXqYhIDMbNX023txeyesNmHFi9YTPd3l7IuPmry/7mGzdChw7QrBk/bNrGX67uzSsNL43vNSJCG2iOtApi3uHH3Ye5e6a7Z9asWWjpDhGRpHvsX1+yeduOXY5t3raDx/71ZdneePx4aNAAhg/nlaZ/4eI2TzC3zknxvUY+yU4K35vZoQCRn+six1ez6wbkh0WOiYikhDUbNsd0vFjr1sFVV0GrVlCjBsydS4+zr2NLpcrxu0Yhkp0UxgM3RH6/AXgn3/HrI7OQzgI2ajxBRFJJrepVYzoelTu8+mrQOhg7Fh5+GLKzITMzftcoQiKnpL4OzAaONbNVZtYO6ANcZGbLgAsjjwH+CfyHYJPx54DbEhWXiEgidGlxLFUrZexyrGqlDLq0OLbkb7JyJfzP/8C118LRR8P8+fDAA7DXXvG7RjESOfvo6ihPNS/kXAduT1QsIiKJljsDqFSzj3buhGHD4L77YMcOGDQI7rgDMnZNAGW6RglZ8Pc4NWVmZrr2UxCRlLZsGdx0E8yYARdeGCSH+vUTekkzm+fumYU9pzIXIiJh2L4d+vWDk0+Gzz+H4cNh0qSEJ4TipPTOayIiKenzz6FdO5g3D7Ky4KmnoFatsKMC1FIQEUmeLVugRw/IzISVK/m471AaN7qD+kPm07jPtLguQistJQURkWSYPRsaNoReveCaa5gwaio3/FKX1Rt/j/8K6DJQUhARSaRNm+Duu6Fx4+D3iRNh5EgenbsuMSugy0hjCiIiiTJlCrRvD8uXw+23Q+/e8Ic/AAlYAR0nSgoiIvG2fj107gwjRsAxxwTTTc89d5dTalWvyupCEkCt6lUTX221COo+EhGJp7FjgxIVI0dC167BTKMCCQGir06+4Liaiau2WgJKCiIi8fD993DlldC6NRxyCHz8cdBdVKVKoadnNaxN79YnUbt6VQyoXb0qvVufxPQvckIda1D3kYhIWbjDyy8Hg8mbNsEjj0CXLlCpUrEvzWpYe7duoU5vfFbouckaa1BSEBGJUW6fP9+u4PGpz3D2V5/AOecEq5KPO65M713UWEMyqPtIRCQG4+avpvuYz2k27S3+Nfx2Tv5mIQ+3uJVxQ0aVOSFAciqhFkUtBRFJSfGaoRPr+7zxyhRefLMfjVYtYUa9hnS/5A5W7XcwtScvI+v0w6O+rqSSUQm1KEoKIpJycvdDzh2QzZ2hA8T0xzOm99m2Dfr358XBPdlcqTL3XtqJMSc2AzMgvn3+hY01JIu6j0Qk5cRrP+QSv8/8+XDmmdCtG7OOP4uL2j3DmJOa5yUESF6ff6IpKYhIyonXauBi3+f33+Fvf4MzzoA1a2D0aDa+9Dq/7n/gLucns88/0dR9JCIpJ14zdIp8n3//Oyhv/eWX0KYNPP44HHAAWZFzYunzD3OFcqyUFEQk5XRpcewuYwFQum/rhb1PjZ1bePmz0dB9JNSpA//6F1x88S6vi6XPP17jH8mi7iMRSTnRVgPH+ke24Pv8ed0iZr58J0e8ORI6doRFi3ZLCLGK1/hHsqilICIpKV4zdLIa1iarblW4556gXtFxx8HomUGp6zgor9VQo1FLQUTS25gxQQG7V14JBpXnz49bQoDo4xzldbaSkoKIpKfvvoPLL4crroDatSE7O9gVLUoBu9IKe4VyrNR9JCKhS+rsHPegm6hTJ9i8Gfr0gXvvhT0T8+cw7BXKsVJSEJFQJXV2zvLl0KEDTJ4c7HHw/PPBJjgJFuYK5Vip+0hEQpWU2Tk7drDgvn/w27HH8+uHH/H4n+5k3MBXk5IQUo1aCiISqoTPzlm6lB+vvp6TP8/mg/qn0/2S21mz70FUHbcY9tgjZb7BJ4uSgoiEKt77B+SOT6z78Rc6LxjPTdNfYc89K9PpsnsYe8IFefWKclsjSgq7UlIQkVDFa3Uy/Hd84siVX/LcxME0WPcNExs0pUez9vywz/67nV9e1wqESUlBREIVz9k5g99bwJ2TR9D+47f5cZ/qdPjz35h0zNlkmAWzjgooqjWSSvWK4klJQURCF5fZOTNmMGLgTdRfv4ZRJ1/Moxe05ecq1QDY4U7VShklbo2kWr2ieApl9pGZdTKzxWa2yMxeN7MqZlbfzOaa2ddm9oaZ7RVGbCKSYn7+GW6/Hc47j71wrvlLL7r+8c68hAD/rY1U0lpJqVavKJ6S3lIws9rAnUADd99sZm8CVwGXAgPdfZSZDQXaAc8kOz4RSSETJ8LNN8OqVXD33Xz6v7cxf+L/QSEtglhaI6lWryiewuo+2hOoambbgL2B74BmwDWR50cCf0dJQSStRe3X//HHYEXyyy8HdYtmzYKzzuJ/gB1V9y7zWEC8Z0SlkqQnBXdfbWaPA98Cm4FJwDxgg7tvj5y2Cij0/0Uz6wB0AKhTp07iAxaRUBTarz9mAbUmvUuj/j1h/Xro2RO6d4fKlfNeF4/xiXjOiEo1YXQf7Q+0AuoDG4C3gEtK+np3HwYMA8jMzNx9OoGIVAgF+/UP+uVHek1+hkbL5kBmJkyZAiefnJBrp1q9ongKo/voQuAbd88BMLO3gcZAdTPbM9JaOAxYHUJsIlJO5PXfu3Plgsk8MH04e+3YxqPnt6X75GcTVsAuVyrVK4qnMJLCt8BZZrY3QfdRcyAbmA5cAYwCbgDeCSE2ESknalWvyh7Lv6HP+0NovGIBcw4/kfv/eCfb6x9J9wQnhHQWxpjCXDMbDXwKbAfmE3QHTQBGmVmvyLHhyY5NRMqJHTsY+sNMjhzRmx22B91b3M7rp7Sgyl6V6J0G/fphCiXduvuDwIMFDv8HaBRCOCJSnixeDO3acdLcuaxt0pxbmrTnc6+WVv36YVIbTETKh61bgw1vevWCffeFV1/lkKuvZlykgJ0kh5KCiITvk0+gXTtYuBCuvhoGD4aaNcOOKi1pkx0RCc9vv0GXLnDWWfDTTzB+PLz2mhJCiNRSEJFwfPABtG8PX38dbJHZrx/st1/YUaU9tRREJLk2boRbboELLgjKWU+bBs8+q4RQTigpiEjyvPcenHACPPcc3HsvLFgQJAcpN9R9JCJAgjeVycmBu+6C11+HE0+Et9+GRpqBXh6ppSAiecXnVm/YjPPfTWXGzS9jtRn3IBE0aACjR8NDD8G8eUoI5ZhaCiJpprAWQVGbypS6tbBqFdx6a9Bl1KgRDB/OuG3789iAj9KuyFwqUUtBJI1EaxEUtncAlHJTmZ07YdiwYOxg6lQYMABmzWLctv0T0xqRuFJSEEkj0VoEGVFWDce8qczXX0Pz5sFuaKefHixG69QJMjLSeovLVKKkIJJGon3zz93YPr+YNpXZvh369w/2N/j002B20dSpcOSRxV47Hba4TCVKCiJpJNo3/1g3tt/FwoVwzjnQuTNcdBEsWQI33QQFWh/Rrp0OW1ymEg00i6SRoraZjHlTmS1b4NFHg3/77w+jRsGVV+6WDEpybSk/lBRE0kjctpmcOzcoYLd4MVx7LQwcCAcemJxrS0KZe+puc5yZmenZ2dlhhyGSPjZtgh49YNAgqF0bhg6Fyy4LOyqJkZnNc/fMwp5TS0FEipS7rqHu53N4bNKT1P7pu2D9QZ8+wb4HRbxGLYLUo6QgIlGNm7+aR1+bTadJz3H1gkn8Z/9aXH9dX1q3/ytZRSSE/GMHuesRACWGFKCkICJRzR44gnfHDuLATRsYeublDGx8DVsqVeb/iljpnJDV0ZI0Sgoisrt16+DOO+n7xhssrVmPm1r3YOGhR+c9XdTaAq1HSG1KCiIpLO599+7w6qtBRdNff2XYRW3od0ortmfs+qeiqLUFtapXLbRshtYjpAYtXhNJUXGvbLpyJbRsCdddB8ccA/Pnc1Dfh6lUpfIupxW3tqBLi2PLtjpaQqWWgkiKilvf/c6dwc5n998PO3YE003vuAMyMsjKd62CrZForRStR0htSgoiKSouffdffRWUpJg5Ey68MKhuWr/+LqcUttK5uBlGMa+OlnJD3UciKaq4WkLj5q+mcZ9p1O86gcZ9pu3arbR9O/TrB6ecEtQuGjECJk3aLSFEo4qnFZeSgkiKKqrvvsjxhs8/hzPPDLqL/vjHoIBdmzZRaxYVRjOMKi51H4mkgKJmGRV2vHGfabt9k9+x+Xc23NsVZo6CAw6At96Cyy+PKRnk0gyjiktJQaScK0n/fUEFv7Gftmopfd8fwtE/roTrrw92Q6tRo9QxqeJpxaXuI5FyrjT997nf2PfeupkHpzzL6Ffvo+q237nnxt4wcmSZEgIEyajU+y9IuaaWgkg5V5r++y4tjuXdx0fy9wlDOHzj97x4WkueaN6GHlefGbe4NMOoYlJSECnnYu6/X7+erCd6kPXaC6yoeTj/+9e+rDkxkx5aKyAloKQgUs7F1H8/dizcdhvk5EC3btTt2ZO3qlRJYrSS6kJJCmZWHXgeOBFwoC3wJfAGUA9YDlzp7uvDiE8kLLHOMsqzdi107AijR8Opp8KECXDaaeHchKS0sFoKg4H33f0KM9sL2BvoDkx19z5m1hXoCtwfUnwiSVeaWUa4w0svQadO8NtvwX7JnTtDpUrJDF0qkKTPPjKz/YCmwHAAd9/q7huAVsDIyGkjIa/sikhaiHmW0YoVweKzG2+EBg3gs8+gWzclBCmTMKak1gdygBfMbL6ZPW9m+wAHu/t3kXPWAgcX9mIz62Bm2WaWnZOTk6SQRRKvxLOMdu6EJ5+EE06Ajz6CJ56AGTPguOOSEKVUdGEkhT2B04Bn3L0hsImgqyiPuzvBWMNu3H2Yu2e6e2bNmjUTHqxIshRXywiAL7+Epk2D8YMmTWDx4qCi6R5aciTxEcYnaRWwyt3nRh6PJkgS35vZoQCRn+tCiE0kNEXuQ7BtG/TuHRSwW7IEXnwRJk6EunXDCVYqrKQPNLv7WjNbaWbHuvuXQHNgSeTfDUCfyM93kh2bSLLENMuIddCoZTBmcMUVQXfRIYeEewNSYYU1+6gj8Gpk5tF/gDYErZY3zawdsAK4MqTYRBKqxLOMfv8dHnoIHnsMDjwQxoyB1q3DClvSRChJwd0/AzILeap5kkMRSajCWgQl2jHto4+gXbtgE5w2baB/f9h//xDuQNKNRqdEEiTangaFlayAyCyjX34JBo7PPRe2bg02vhkxQglBkkZlLkQSJFqLIMOMHb775LqsdYvgxNtg5Uq480545BGoVi1Z4YoASgoiCRNt3cEOd6pWyshLGPtt/oWHpj9P1sKpwVqDjz6Cc85JZqgiedR9JJIg0dYd5O49UHu/Klz6xUdMH3Erf1r6IfztbzB/vhKChEpJQSRBilp3kHXIHvz7k6d4+p0+HHDMEezxySfQqxeooqmETN1HIglS6LqDi48h67NJcM89wZTTvn2D3/fUf4pSPuiTKJJAu6w7+OYb6NAGpkwJZhc9/zwcc0y4AYoUoO4jkUTbsQOGDIETT4Q5c+Dpp+GDD5QQpFxSS0EkkZYuDRahzZ4dlLkeOhTq1Ak7KpGo1FIQSYRt24KB41NPDSqbvvxysBuaEoKUc8UmBTPraGZaTilSUvPmQWYm9OgBf/5z0Fq49lowCzsykWKVpKVwMPCJmb1pZpeY6ZMtUqjNm+H++6FRI8jJgXHjYNQoOOigsCMTKbFik4K7PwAcTbB95o3AMjN71MyOTHBsIqljxoxgr4N+/aBt22DPg1atwo5KJGYlGmh2dzeztQTbZG4H9gdGm9lkd78vkQGKJFtRex3s5uefoWtXeOYZqF8/mG7aXMV+JXUVmxTM7C7geuAH4Hmgi7tvM7M9gGWAkoJUGMXtdbCLf/4Tbr4ZVq+GTp3g4Ydhn32SHbJIXJVkTOEAoLW7t3D3t9x9G4C77wRaJjQ6kSQraq+DPD/8EAwcX3YZ7LsvH74wjsYHtaT+wx/QuM80xs1fndygReKoJGMKD7r7iijPLY1/SCLhiVbZdM2GzeAOb7wBDRoEP3v2ZPyId7nl67122zNBiUFSldYpiOQTrbLpyXtsgqwsuOoqqFsXPv0UHnqIvtOXF9+yEEkhSgoi+exW2dSd6xZNZvQT7YNd0B5/PFidfNJJQDEtC5EUpDIXIvnkr2yasfwbBkx5isz/mw/nnRcUsDvqqF3Or1W9aqHba0ZrcYiUd2opiBSQdfIh/Lvy58x4qSOZ676GZ5+FadN2SwhQ9J4JIqlILQWR/BYtCgrYffxxMLto6FA47LCopxe6Z0JR6xpEyjklBRGArVuhd2945BHYbz947bVgULkEVV122TNBJMUpKYh88klQmmLRIrjmGhg0CGrWDDsqkVBoTEHS1ruzvua1pley48yzWPftWuYMfAFefVUJQdKakoKkpY+GvcnJLZtyzcy3GHXKxTRv+xRtfjxEi84k7an7SNLLxo1w3300GTaM5dUP5eqrHmV23ZOD5yKLzqKND8RUKE8kRSkpSPp491245RZYu5ZhjVozoMk1/F6pyi6nRFt0FlOhPJEUpu4jqfhycoIB5D/9CWrUgDlzGPnn23dLCBB90VmJCuWJVABKClJxuQdTS48/HkaPhoceguxsOOOMmBedqZyFpAt1H0nFtGoV3HorvPcenHkmDB8OJ5yQ93Ssi85UzkLShZKCVCw7d8Jzz0GXLrB9OwwYAHfeCRkZu50ay6KzLi2O3WVMAVTOQiqm0JKCmWUA2cBqd29pZvWBUUANYB5wnbtvDSs+SUHLlkH79vDhh9CsWZAcjjgiLrOGVM5C0kWYLYW7gKXAvpHHfYGB7j7KzIYC7YBnwgpOUsj27cEq5B49oHLloJpp27ZgFtdZQypnIekglIFmMzsMuIxgz2fMzIBmwOjIKSOBrDBikxSzYAGcfXbQXdSiBSxZEhS0i9Qs0qwhkdiENftoEHAfsDPyuAawwd23Rx6vAgr9SmZmHcws28yyc3JyEh6olFNbtsCDD8Lpp8OKFcH2mGPHQq1au5ymWUMisUl695GZtQTWufs8Mzs/1te7+zBgGEBmZqbHNzpJhLivBJ4zJ2gNLFkC114bdB3VqFHoqZo1JBKbMFoKjYE/mdlygoHlZsBgoLqZ5SapwwAVoakAcvv047Kx/aZN0KkTnHMO/PwzTJgAL78cNSGANsERiVXSWwru3g3oBhBpKXR297+a2VvAFQSJ4gbgnWTHJvFXVJ9+Ua2Fgq2LvtVzaNKvG3zzTbD+oE8f2HffqK/PpVlDIrEpT+sU7gdGmVkvYD4wPOR4JA5K06eff8bQvr//SsfXh9BkwSR+rVOfah9+CE2bxhSDZg2JlFyoScHdPwA+iPz+H6BRmPFI2RQ2dlCaPv3c1sVFy+bQa9LT1Ni0gWfOvII3L2vL9BgTgojEpjy1FCSFRVsPcPnptRkzb3VMK4G3rl7Dk1OG0fKLmSw5qD7tLu/JokOOwjbtjPoaEYkPJQWJi2hjB9O/yKF365NK1qfvDq+8wpTht1Nl62YeO/c6nj3zcrZnBB9TzRgSSTwlBYmLosYOStSn/+23wV4HEyey/eTTaX1OBxbv99/XaMaQSHKodLbERbRv8cV+u9+5E55+Oqhg+uGHMHgwNT6dS/sOl1G7elUMqF29Kr1bn6TBYpEkUEtB4qJUVUS/+gpuuglmzoSLLoJnn4X69QHNGBIJi5KCxEVM6wG2b4f+/YMyFVWrwgsvwA035NUrKg3tnywSH0oKEjcl+nb/+edBBdNPP4U//xmeegoOPbRM19X+ySLxo6QgyfH779CrF/TtG5SlGD0aLr885rcprEVQ2lXTIrI7JQVJvFmzggJ2X3wRdBMNGAAHHBDz20RrERRMCLlUCVUkdpp9JInz66/BVphNmsBvv8H778OLL5YqIUD0tRAZUcYitK5BJHZKCpIYkybBiSfCk0/C7bfDokXBJjhlEO2b/w53VUIViRMlBYmv9euhTZsgAVSpAjNmwBNPwB/+UOa3jvbNP3cdg9Y1iJSdxhQkft5+O2gV5ORAt27Qs2eQGOKkqLUQWtcgEh9KClJ2a9fCHXfAmDFw6qnwz39Cw4Ylemks6wu0N4JI4ikpSOm5w8iRcM89wUDyo49C585QqVKJXl6a9QVqEYgklpKCRP22XuS3+OXL4eabgwHlxo3h+efhuONiuq7WF4iUP0oKaS7at/XsFT/tsg9C3rf4nTvJmjUuGDMwC2YX3Xor7BH7nIXS7MomIomlpJDmon1bf33uSna473K81trl1L+8C6yITC999lmoW7fU1y7Nrmwikliakprmipr7n2vPHdu5bfab/POFjtT9fnkwjjBxYpkSAgSzibS+QKR8UUshzUX7tp5hxg53Tlj7Nf0mDuGEdf9hwrGNeebyu3nv+ivicm3NJhIpf5QU0ly0uf9/ObEGtYY8TtvZo/lp7/24Oas7M048l96tT4rr9TWbSKR8UVJIE9FmEhX2bb33QRtp2vNa+Oor3s28hAca30i1Q2rSW9/iRSo88wKDiakkMzPTs7Ozww6j3Cs4wwiC1sBupSB++SWYVfTUU1CvHjz3HFx4YfIDFpGEMrN57p5Z2HMaaE4DRa0HyDNxYrBP8tNPw113wcKFSggiaUhJIQ0UuR7gxx/h+uvh0kuhWjX4979h0KDgdxFJOxpTSFGx1AwqdIaRO39d9Qk0aAs//QQPPBD8q1w5CdGLSHmlpJCCYq0ZVHCGUc1ff6L3lGe48MvZcPrpQamKU05J3g2ISLmlpJCCihsjiNaCeOz9L2g8czw9pg9nb98O/fpBp06wpz4GIhLQX4MUFG2MoOCexbu0IKpvJWtaH5gyBZo2DWYWHXNM0mIWkdSgpJCCilqFXLAFsWXLVpb3fBSmvQgZGfDMM9ChQ6kK2OWKZTxDRFKLZh+loGg1gwoWsDvqh28Z/ep93P3e03DeebB4MdxyS5kTQre3F7J6w2ac/7ZGxs1fXer3FJHyI+lJwcwON7PpZrbEzBab2V2R4weY2WQzWxb5uX+yY0sVWQ1rF7once1IddFKO7bR8d+vM+HFO6m3/jseurIbTJgAhx9e5muXaM2DiKSsMLqPtgP3uvunZvYHYJ6ZTQZuBKa6ex8z6wp0Be4PIb6UEK1m0CtPjuHhdwdyfM5yxh/flL6X3EqX684N9j6IA+2BIFKxJT0puPt3wHeR338xs6VAbaAVcH7ktJHABygplNzmzWS9PphWL/bnh2r70751D5accX7c+/u1B4JIxRbqQLOZ1QMaAnOBgyMJA2AtcHCU13QAOgDUqVMnCVGmgA8/hJtugq+/xtq3p2a/fjxXvXpCLhWtqqr2QBCpGEIbaDazasAY4G53/zn/cx5U6Su0Up+7D3P3THfPrFmzZhIiLcd+/jnYCvP882HnTpg6FYYNgwQlBIg+nqHZRyIVQygtBTOrRJAQXnX3tyOHvzezQ939OzM7FFgXRmwpY8KEYCbRmjVwzz3wj3/APvsk5dLaA0Gk4gpj9pEBw4Gl7j4g31PjgRsiv98AvJPs2FLCDz/AtddCy5aw774waxb075+0hCAiFVsYLYXGwHXAQjP7LHKsO9AHeNPM2gErgCtDiC00xS4Ic4c33oCOHWHjRnjwwWDvAxWwE5E4CmP20UdAtPmRzZMZS3lRbIG71avhtttg/Hg44wwYPhxOiu+2mCIioBXN5ULUBWHvfxHUKGrQACZPhscfh9mzlRBEJGFU+6gcKGzhV53139Hn9Sfg2wXB7KLnnoOjjkp+cCKSVpQUyiBeheHyLwjbY+cO2mSPp/PMV9iRsSc8+2ywBqEM9YpEREpKf2lKKZ6F4XIL3B2Ts5y3X+lCj+nDmVP/FD4aN73MFU1FRGKhvzalFM/CcFkn1OTtnMlMGHk3h29YS8+r/sbGUWO45JIz4hWuiEiJqPuolOJWGO7jj6FdO45ftAiuuYYagwfzjwMPjEOEIiKxU1IopaIKw5VorOG336BHDxg0CA49FN59N1iQJiISInUflVK0jW4uOK5m8WMN06cH00oHDID27YPNb5QQRKQcUEshn1hmE+UeL3h+UWMNWUdUgy5dgumlRx4ZJIfzz49bTCIiZaWkEFHsquJCFFYYrtMbnxV67vHZH0CD62DtWujcGR56CPbeO+4xiYiUhbqPIuI1m6jgZjMH/LaRIeP78fyYh6FGDZgzBx57rNiEEM+YRERKSkkhIl6zifLGGtxptXg6U56/lUu+nMXSWztDdnZQuyjJMYmIlJS6jyLitc1kVsPaVF27hmqdOtL4y7ksPvx41g54kuZXNAstJhGRklJLISLabKKYtpncuROGDqXFX5rTeOVCGDiQE75ZWKqEELeYRERioJZCRLTZRFkNa5dsBtCyZcH00g8/hObNg20xjzhil1NinUlUVEwiIolgwXbIqSkzM9Ozs7MTeo2CM4Ag+Laety/x9u0wcCD07BlseNO/P7RtC2axvY+ISJKY2Tx3zyzsOXUfFaPIGUALFsDZZ8N990GLFrBkCbRrt1tCKPZ9RETKCSWFYhQ202ev7dv4y7vPwemnw7ffwptvwtixUKtWTO9T1HERkTBoTKEYBWcAnbZ6KX0nDuHoH1fCddcFXUc1asT8PvmPi4iUF2opFCN3BlDVrb/Tc8owRr9yH/ts+51ZT7wEL71UooSQ/33y00wiESlv1FIoRlbD2tScO5P63TtRa/1axpz1Jyr360vLc4+L+X1AM4lEpHxTUijKhg1w7700HjECjj4axr3B5U2blvrtCquVJCJSnqRdUijxWoFx4+C222DdOujaNZhyWlX9/yJSsaVVUihR1dHvv4eOHeGtt+CUU4LNb04/PayQRUSSKq0GmotcK+AeDBwffzy88w488gh88okSgoiklbRqKURdE/DtCrj0Unj//WAx2vDhQXIQEUkzaZUUCq4VMN/JtfP/SbcPR0KlPWDIkGAcISOjiHcREam40qr7KP9agSN+XMUbr3Xl4clD+fW0M2DRomAsQQlBRNJYWrUUcgeTlzwyiHvHDWZLpcp8+vcBnNbz7kLrFYmIpJu0SgoQWStwdyvgGyo/+SSnHXJI2CGJiJQbaZcUAGjSJPgnIiK7SKsxBRERKVq5SgpmdomZfWlmX5tZ17DjERFJN+UmKZhZBvAU8EegAXC1mTUINyoRkfRSbpIC0Aj42t3/4+5bgVFAq5BjEhFJK+UpKdQGVuZ7vCpybBdm1sHMss0sOycnJ2nBiYikg/KUFErE3Ye5e6a7Z9asWTPscEREKpTylBRWA4fne3xY5JiIiCRJeUoKnwBHm1l9M9sLuAoYH3JMIiJpxdw97BjymNmlwCAgAxjh7o8Uc34OsKKYtz0Q+CEuAaYW3Xd6Sdf7hvS997Lcd113L7T/vVwlhUQws2x3zww7jmTTfaeXdL1vSN97T9R9l6fuIxERCZmSgoiI5EmHpDAs7ABCovtOL+l635C+956Q+67wYwoiIlJy6dBSEBGRElJSEBGRPBU6KaRLKW4zG2Fm68xsUb5jB5jZZDNbFvm5f5gxJoKZHW5m081siZktNrO7Iscr9L2bWRUz+9jMPo/c90OR4/XNbG7k8/5GZBFohWNmGWY238zeizyu8PdtZsvNbKGZfWZm2ZFjCfmcV9ikkGaluF8ELilwrCsw1d2PBqZGHlc024F73b0BcBZwe+T/44p+71uAZu5+CnAqcImZnQX0BQa6+1HAeqBdeCEm1F3A0nyP0+W+L3D3U/OtTUjI57zCJgXSqBS3u88AfipwuBUwMvL7SCArmTElg7t/5+6fRn7/heAPRW0q+L174NfIw0qRfw40A0ZHjle4+wYws8OAy4DnI4+NNLjvKBLyOa/ISaFEpbgrsIPd/bvI72uBg8MMJtHMrB7QEJhLGtx7pAvlM2AdMBn4P2CDu2+PnFJRP++DgPuAnZHHNUiP+3ZgkpnNM7MOkWMJ+ZzvGY83kfLN3d3MKuzcYzOrBowB7nb3n4Mvj4GKeu/uvgM41cyqA2OB48KNKPHMrCWwzt3nmdn5IYeTbE3cfbWZHQRMNrMv8j8Zz895RW4ppHsp7u/N7FCAyM91IceTEGZWiSAhvOrub0cOp8W9A7j7BmA6cDZQ3cxyv+hVxM97Y+BPZracoDu4GTCYin/fuPvqyM91BF8CGpGgz3lFTgrpXop7PHBD5PcbgHdCjCUhIv3Jw4Gl7j4g31MV+t7NrGakhYCZVQUuIhhPmQ5cETmtwt23u3dz98PcvR7Bf8/T3P2vVPD7NrN9zOwPub8DFwOLSNDnvEKvaI61FHeqMrPXgfMJSul+DzwIjAPeBOoQlBe/0t0LDkanNDNrAswEFvLfPubuBOMKFfbezexkgoHFDIIvdm+6+z/M7AiCb9AHAPOBa919S3iRJk6k+6izu7es6Pcdub+xkYd7Aq+5+yNmVoMEfM4rdFIQEZHYVOTuIxERiZGSgoiI5FFSEBGRPEoKIiKSR0lBRETyKCmIiEgeJQUREcmjpCASR2Z2hpktiOx5sE9kv4MTw45LpKS0eE0kzsysF1AFqAqscvfeIYckUmJKCiJxFqm19QnwO3BOpKKpSEpQ95FI/NUAqgF/IGgxiKQMtRRE4szMxhMUaKsPHOrud4QckkiJaZMdkTgys+uBbe7+WmSf8Flm1szdp4Udm0hJqKUgIiJ5NKYgIiJ5lBRERCSPkoKIiORRUhARkTxKCiIikkdJQURE8igpiIhInv8HyppPu7zB1ywAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X.numpy(), y.numpy())\n",
    "plt.plot(x1,y1,'r')\n",
    "plt.title('Current Model')\n",
    "plt.ylabel('y')\n",
    "plt.xlabel('x');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Great job!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
